2023-07-31 09:53:11,469 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-07-31 09:53:11,816 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-07-31 09:53:11,934 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-07-31 09:53:11,992 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-07-31 09:53:11,999 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-07-31 09:53:12,016 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=345, ProcessCount=174, AvailableMemoryMB=2487
2023-07-31 09:53:12,019 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-07-31 09:53:12,019 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-07-31 09:53:12,178 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1017727011:84 , srv1032136202:84 , srv1054903096:84 , srv1079755153:84 , srv1082618739:84 , srv1128773230:84 , srv1148769110:84 , srv1171135551:84 , srv1174342627:84 , srv120410829:84 , srv12043243:84 , srv120811902:84 , srv1221950223:84 , srv1246686271:84 , srv1247193177:84 , srv1253281314:84 , srv1269737787:84 , srv1311571387:84 , srv1331114204:84 , srv1355135185:84 , srv1407033576:84 , srv143628053:84 , srv1510846052:84 , srv1536972090:84 , srv1561874147:84 , srv1592356482:84 , srv1592432967:84 , srv1633403704:84 , srv1637748916:84 , srv1653953394:84 , srv1655264648:84 , srv1671276607:84 , srv1678285690:84 , srv169750908:84 , srv1700575128:84 , srv1704950811:84 , srv1712137209:84 , srv1713968874:84 , srv1715870700:84 , srv1786596284:84 , srv1793348648:84 , srv1804615491:84 , srv1848521095:84 , srv1854450003:84 , srv1864844699:84 , srv1864998393:84 , srv1949166327:84 , srv1951955274:84 , srv1978559903:84 , srv1980700582:84 , srv2009322043:84 , srv2021804248:84 , srv2050221444:84 , srv20584033:84 , srv2077540095:84 , srv207781747:84 , srv208051001:84 , srv2114180218:84 , srv2117810007:84 , srv2132355115:84 , srv215407768:84 , srv22239237:84 , srv22678136:84 , srv241279565:84 , srv267337040:84 , srv28493324:84 , srv298348055:84 , srv30924677:84 , srv366048020:84 , srv36935983:84 , srv388184788:84 , srv41831043:84 , srv470947714:84 , srv497608842:84 , srv510503973:84 , srv517147999:84 , srv521811578:84 , srv55095226:84 , srv587962583:84 , srv610558355:84 , srv628223570:84 , srv641392705:84 , srv64639389:84 , srv671034712:84 , srv704304235:84 , srv715598173:84 , srv752056354:84 , srv76563153:84 , srv853028547:84 , srv864497382:84 , srv879299439:84 , srv911308893:84 , srv921649236:84 , srv92293981:84 , srv9248328:84 , srv947429201:84 , srv955435481:84 , srv967803605:84 , srv987590192:84 , srv1399563722:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-07-31 09:53:12,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-31 09:53:12,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2132355115=60, srv366048020=69, srv1637748916=29, srv2021804248=52, srv1713968874=38, srv388184788=71, srv497608842=74, srv1221950223=12, srv1079755153=3, srv1536972090=24, srv22239237=62, srv2117810007=59, srv1128773230=5, srv1678285690=33, srv517147999=76, srv298348055=67, srv752056354=87, srv1054903096=2, srv28493324=66, srv30924677=68, srv1331114204=18, srv41831043=72, srv2009322043=51, srv853028547=89, srv947429201=96, srv955435481=97, srv967803605=98, srv1864998393=46, srv987590192=99, srv64639389=83, srv215407768=61, srv36935983=70, srv1253281314=15, srv1854450003=44, srv1655264648=31, srv587962583=79, srv267337040=65, srv1355135185=19, srv208051001=57, srv92293981=94, srv2077540095=55, srv1247193177=14, srv76563153=88, srv120410829=9, srv241279565=64, srv1510846052=23, srv120811902=11, srv1311571387=17, srv2050221444=53, srv1174342627=8, srv20584033=54, srv169750908=34, srv1671276607=32, srv864497382=90, srv1864844699=45, srv1171135551=7, srv1592356482=26, srv12043243=10, srv628223570=81, srv704304235=85, srv1786596284=40, srv521811578=77, srv911308893=92, srv143628053=22, srv1017727011=0, srv879299439=91, srv22678136=63, srv1978559903=49, srv1592432967=27, srv55095226=78, srv9248328=95, srv1407033576=21, srv1848521095=43, srv1633403704=28, srv1653953394=30, srv641392705=82, srv1712137209=37, srv1980700582=50, srv1082618739=4, srv1246686271=13, srv1269737787=16, srv1804615491=42, srv1704950811=36, srv1399563722=20, srv510503973=75, srv1032136202=1, srv1561874147=25, srv610558355=80, srv671034712=84, srv1715870700=39, srv2114180218=58, srv1949166327=47, srv715598173=86, srv1148769110=6, srv1951955274=48, srv470947714=73, srv1700575128=35, srv1793348648=41, srv207781747=56, srv921649236=93} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-31 09:53:12,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-31 09:53:12,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-31 09:53:12,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-31 09:53:12,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-31 09:53:12,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-31 09:53:12,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-31 09:53:12,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-31 09:53:12,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-31 09:53:12,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-31 09:53:12,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-31 09:53:12,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-31 09:53:12,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-31 09:53:12,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-31 09:53:12,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-31 09:53:12,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-31 09:53:12,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-31 09:53:12,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-31 09:53:12,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-31 09:53:12,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-31 09:53:12,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-31 09:53:12,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-31 09:53:12,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-31 09:53:12,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-31 09:53:12,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-31 09:53:12,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-31 09:53:12,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-31 09:53:12,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-31 09:53:12,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-31 09:53:12,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-31 09:53:12,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-31 09:53:12,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-31 09:53:12,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-31 09:53:12,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-31 09:53:12,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-31 09:53:12,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-31 09:53:12,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-31 09:53:12,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-31 09:53:12,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-31 09:53:12,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-31 09:53:12,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-31 09:53:12,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-31 09:53:12,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-31 09:53:12,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-31 09:53:12,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-31 09:53:12,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-31 09:53:12,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-31 09:53:12,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-31 09:53:12,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-31 09:53:12,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-31 09:53:12,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-31 09:53:12,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-31 09:53:12,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-31 09:53:12,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-31 09:53:12,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-31 09:53:12,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-31 09:53:12,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-31 09:53:12,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-31 09:53:12,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-31 09:53:12,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-31 09:53:12,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-31 09:53:12,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-31 09:53:12,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-31 09:53:12,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-31 09:53:12,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-31 09:53:12,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-31 09:53:12,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-31 09:53:12,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-31 09:53:12,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-31 09:53:12,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-31 09:53:12,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-31 09:53:12,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-31 09:53:12,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-31 09:53:12,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-31 09:53:12,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-31 09:53:12,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-31 09:53:12,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-31 09:53:12,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-31 09:53:12,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-31 09:53:12,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-31 09:53:12,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-31 09:53:12,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-31 09:53:12,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-31 09:53:12,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-31 09:53:12,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-31 09:53:12,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-31 09:53:12,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-31 09:53:12,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-31 09:53:12,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-31 09:53:12,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-31 09:53:12,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-31 09:53:12,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-31 09:53:12,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-31 09:53:12,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-31 09:53:12,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-31 09:53:12,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-31 09:53:12,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-31 09:53:12,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-31 09:53:12,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-31 09:53:12,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-31 09:53:12,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-31 09:53:12,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-31 09:53:12,508 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-31 09:53:12,508 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-07-31 09:53:12,508 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-07-31 09:53:12,915 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 603 ms to try 5000 different iterations.  Found a solution that moves 4662 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17980023640376244. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0389005914731347, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05446608516518421, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.518, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0389005914731347, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17541666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23783333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-31 09:53:12,960 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv12043243:83 , srv1253281314:83 , srv2077540095:83 , srv1079755153:84 , srv1561874147:84 , srv1633403704:84 , srv169750908:84 , srv1713968874:84 , srv1864998393:84 , srv1949166327:84 , srv22239237:84 , srv22678136:84 , srv41831043:84 , srv517147999:84 , srv587962583:84 , srv610558355:84 , srv628223570:84 , srv641392705:84 , srv1032136202:85 , srv1054903096:85 , srv1171135551:85 , srv143628053:85 , srv1592432967:85 , srv1678285690:85 , srv1712137209:85 , srv1786596284:85 , srv1804615491:85 , srv1978559903:85 , srv2009322043:85 , srv2050221444:85 , srv2117810007:85 , srv64639389:85 , srv76563153:85 , srv864497382:85 , srv92293981:85 , srv947429201:85 , srv1082618739:86 , srv1128773230:86 , srv1592356482:86 , srv1637748916:86 , srv1653953394:86 , srv1864844699:86 , srv1951955274:86 , srv1980700582:86 , srv208051001:86 , srv215407768:86 , srv298348055:86 , srv366048020:86 , srv36935983:86 , srv671034712:86 , srv955435481:86 , srv120811902:87 , srv1246686271:87 , srv1269737787:87 , srv1355135185:87 , srv1510846052:87 , srv1655264648:87 , srv1671276607:87 , srv1700575128:87 , srv1704950811:87 , srv1715870700:87 , srv1793348648:87 , srv2132355115:87 , srv28493324:87 , srv388184788:87 , srv497608842:87 , srv521811578:87 , srv704304235:87 , srv120410829:88 , srv1247193177:88 , srv1311571387:88 , srv1331114204:88 , srv1407033576:88 , srv1536972090:88 , srv1848521095:88 , srv1854450003:88 , srv2021804248:88 , srv241279565:88 , srv267337040:88 , srv55095226:88 , srv715598173:88 , srv921649236:88 , srv9248328:88 , srv967803605:88 , srv987590192:88 , srv1017727011:89 , srv1148769110:89 , srv1221950223:89 , srv20584033:89 , srv30924677:89 , srv752056354:89 , srv853028547:89 , srv879299439:89 , srv1174342627:90 , srv2114180218:90 , srv911308893:90 , srv470947714:91 , srv510503973:93 , srv207781747:94 , srv1399563722:436 }
2023-07-31 09:53:12,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-31 09:53:12,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2132355115=60, srv366048020=69, srv1637748916=29, srv2021804248=52, srv1713968874=38, srv388184788=71, srv497608842=74, srv1221950223=12, srv1079755153=3, srv1536972090=24, srv22239237=62, srv2117810007=59, srv1128773230=5, srv1678285690=33, srv517147999=76, srv298348055=67, srv752056354=87, srv1054903096=2, srv28493324=66, srv30924677=68, srv1331114204=18, srv41831043=72, srv2009322043=51, srv853028547=89, srv947429201=96, srv955435481=97, srv967803605=98, srv1864998393=46, srv987590192=99, srv64639389=83, srv215407768=61, srv36935983=70, srv1253281314=15, srv1854450003=44, srv1655264648=31, srv587962583=79, srv267337040=65, srv1355135185=19, srv208051001=57, srv92293981=94, srv2077540095=55, srv1247193177=14, srv76563153=88, srv120410829=9, srv241279565=64, srv1510846052=23, srv120811902=11, srv1311571387=17, srv2050221444=53, srv1174342627=8, srv20584033=54, srv169750908=34, srv1671276607=32, srv864497382=90, srv1864844699=45, srv1171135551=7, srv1592356482=26, srv12043243=10, srv628223570=81, srv704304235=85, srv1786596284=40, srv521811578=77, srv911308893=92, srv143628053=22, srv1017727011=0, srv879299439=91, srv22678136=63, srv1978559903=49, srv1592432967=27, srv55095226=78, srv9248328=95, srv1407033576=21, srv1848521095=43, srv1633403704=28, srv1653953394=30, srv641392705=82, srv1712137209=37, srv1980700582=50, srv1082618739=4, srv1246686271=13, srv1269737787=16, srv1804615491=42, srv1704950811=36, srv1399563722=20, srv510503973=75, srv1032136202=1, srv1561874147=25, srv610558355=80, srv671034712=84, srv1715870700=39, srv2114180218=58, srv1949166327=47, srv715598173=86, srv1148769110=6, srv1951955274=48, srv470947714=73, srv1700575128=35, srv1793348648=41, srv207781747=56, srv921649236=93} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-31 09:53:12,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-31 09:53:12,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-31 09:53:12,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-31 09:53:12,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-31 09:53:12,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-31 09:53:12,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-31 09:53:12,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-31 09:53:12,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-31 09:53:13,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-31 09:53:13,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-31 09:53:13,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-31 09:53:13,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-31 09:53:13,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-31 09:53:13,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-31 09:53:13,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-31 09:53:13,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-31 09:53:13,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-31 09:53:13,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-31 09:53:13,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-31 09:53:13,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-31 09:53:13,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-31 09:53:13,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-31 09:53:13,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-31 09:53:13,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-31 09:53:13,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-31 09:53:13,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-31 09:53:13,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-31 09:53:13,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-31 09:53:13,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-31 09:53:13,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-31 09:53:13,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-31 09:53:13,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-31 09:53:13,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-31 09:53:13,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-31 09:53:13,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-31 09:53:13,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-31 09:53:13,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-31 09:53:13,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-31 09:53:13,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-31 09:53:13,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-31 09:53:13,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-31 09:53:13,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-31 09:53:13,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-31 09:53:13,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-31 09:53:13,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-31 09:53:13,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-31 09:53:13,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-31 09:53:13,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-31 09:53:13,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-31 09:53:13,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-31 09:53:13,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-31 09:53:13,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-31 09:53:13,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-31 09:53:13,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-31 09:53:13,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-31 09:53:13,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-31 09:53:13,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-31 09:53:13,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-31 09:53:13,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-31 09:53:13,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-31 09:53:13,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-31 09:53:13,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-31 09:53:13,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-31 09:53:13,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-31 09:53:13,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-31 09:53:13,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-31 09:53:13,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-31 09:53:13,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-31 09:53:13,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-31 09:53:13,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-31 09:53:13,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-31 09:53:13,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-31 09:53:13,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-31 09:53:13,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-31 09:53:13,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-31 09:53:13,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-31 09:53:13,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-31 09:53:13,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-31 09:53:13,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-31 09:53:13,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-31 09:53:13,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-31 09:53:13,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-31 09:53:13,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-31 09:53:13,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-31 09:53:13,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-31 09:53:13,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-31 09:53:13,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-31 09:53:13,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-31 09:53:13,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-31 09:53:13,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-31 09:53:13,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-31 09:53:13,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-31 09:53:13,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-31 09:53:13,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-31 09:53:13,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-31 09:53:13,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-31 09:53:13,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-31 09:53:13,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-31 09:53:13,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-31 09:53:13,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-31 09:53:13,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-31 09:53:13,184 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0389005914731347, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05446608516518421, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0389005914731347, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17541666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23783333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-31 09:53:13,184 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-07-31 09:53:13,185 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17976759238204776, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0389005914731347, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05446608516518421, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0389005914731347, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17541666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23783333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-07-31 09:53:13,581 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 542 ms to try 5000 different iterations.  Found a solution that moves 2652 regions; Going from a computed imbalance of 0.17976759238204776 to a new imbalance of 0.015271311622936353. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02274250642965898); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.032456480402067424, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2946666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02274250642965898); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012083333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04575, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-31 09:53:13,591 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv207781747:78 , srv20584033:81 , srv2114180218:81 , srv510503973:81 , srv879299439:81 , srv1017727011:82 , srv120410829:82 , srv120811902:82 , srv1536972090:82 , srv1848521095:82 , srv911308893:82 , srv9248328:82 , srv1174342627:83 , srv1221950223:83 , srv1655264648:83 , srv1854450003:83 , srv241279565:83 , srv521811578:83 , srv704304235:83 , srv1082618739:84 , srv1148769110:84 , srv1269737787:84 , srv1331114204:84 , srv1592356482:84 , srv1592432967:84 , srv1715870700:84 , srv215407768:84 , srv298348055:84 , srv752056354:84 , srv955435481:84 , srv967803605:84 , srv987590192:84 , srv1247193177:85 , srv1311571387:85 , srv1407033576:85 , srv1678285690:85 , srv1700575128:85 , srv1864844699:85 , srv1980700582:85 , srv2021804248:85 , srv2117810007:85 , srv2132355115:85 , srv267337040:85 , srv28493324:85 , srv30924677:85 , srv366048020:85 , srv497608842:85 , srv853028547:85 , srv921649236:85 , srv947429201:85 , srv1054903096:86 , srv1171135551:86 , srv1246686271:86 , srv1355135185:86 , srv1510846052:86 , srv1633403704:86 , srv1637748916:86 , srv1671276607:86 , srv1704950811:86 , srv1793348648:86 , srv1978559903:86 , srv36935983:86 , srv610558355:86 , srv64639389:86 , srv715598173:86 , srv92293981:86 , srv1032136202:87 , srv1128773230:87 , srv1653953394:87 , srv1712137209:87 , srv1713968874:87 , srv1804615491:87 , srv22239237:87 , srv22678136:87 , srv388184788:87 , srv41831043:87 , srv470947714:87 , srv641392705:87 , srv671034712:87 , srv1079755153:88 , srv1253281314:88 , srv143628053:88 , srv169750908:88 , srv1786596284:88 , srv1949166327:88 , srv1951955274:88 , srv2050221444:88 , srv517147999:88 , srv55095226:88 , srv628223570:88 , srv76563153:88 , srv864497382:88 , srv1864998393:89 , srv2077540095:89 , srv12043243:90 , srv1561874147:90 , srv2009322043:90 , srv208051001:90 , srv587962583:92 , srv1399563722:540 }
2023-07-31 09:53:13,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-31 09:53:13,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2132355115=60, srv366048020=69, srv1637748916=29, srv2021804248=52, srv1713968874=38, srv388184788=71, srv497608842=74, srv1221950223=12, srv1079755153=3, srv1536972090=24, srv22239237=62, srv2117810007=59, srv1128773230=5, srv1678285690=33, srv517147999=76, srv298348055=67, srv752056354=87, srv1054903096=2, srv28493324=66, srv30924677=68, srv1331114204=18, srv41831043=72, srv2009322043=51, srv853028547=89, srv947429201=96, srv955435481=97, srv967803605=98, srv1864998393=46, srv987590192=99, srv64639389=83, srv215407768=61, srv36935983=70, srv1253281314=15, srv1854450003=44, srv1655264648=31, srv587962583=79, srv267337040=65, srv1355135185=19, srv208051001=57, srv92293981=94, srv2077540095=55, srv1247193177=14, srv76563153=88, srv120410829=9, srv241279565=64, srv1510846052=23, srv120811902=11, srv1311571387=17, srv2050221444=53, srv1174342627=8, srv20584033=54, srv169750908=34, srv1671276607=32, srv864497382=90, srv1864844699=45, srv1171135551=7, srv1592356482=26, srv12043243=10, srv628223570=81, srv704304235=85, srv1786596284=40, srv521811578=77, srv911308893=92, srv143628053=22, srv1017727011=0, srv879299439=91, srv22678136=63, srv1978559903=49, srv1592432967=27, srv55095226=78, srv9248328=95, srv1407033576=21, srv1848521095=43, srv1633403704=28, srv1653953394=30, srv641392705=82, srv1712137209=37, srv1980700582=50, srv1082618739=4, srv1246686271=13, srv1269737787=16, srv1804615491=42, srv1704950811=36, srv1399563722=20, srv510503973=75, srv1032136202=1, srv1561874147=25, srv610558355=80, srv671034712=84, srv1715870700=39, srv2114180218=58, srv1949166327=47, srv715598173=86, srv1148769110=6, srv1951955274=48, srv470947714=73, srv1700575128=35, srv1793348648=41, srv207781747=56, srv921649236=93} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-31 09:53:13,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-31 09:53:13,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-31 09:53:13,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-31 09:53:13,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-31 09:53:13,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-31 09:53:13,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-31 09:53:13,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-31 09:53:13,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-31 09:53:13,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-31 09:53:13,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-31 09:53:13,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-31 09:53:13,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-31 09:53:13,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-31 09:53:13,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-31 09:53:13,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-31 09:53:13,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-31 09:53:13,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-31 09:53:13,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-31 09:53:13,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-31 09:53:13,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-31 09:53:13,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-31 09:53:13,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-31 09:53:13,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-31 09:53:13,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-31 09:53:13,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-31 09:53:13,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-31 09:53:13,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-31 09:53:13,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-31 09:53:13,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-31 09:53:13,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-31 09:53:13,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-31 09:53:13,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-31 09:53:13,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-31 09:53:13,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-31 09:53:13,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-31 09:53:13,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-31 09:53:13,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-31 09:53:13,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-31 09:53:13,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-31 09:53:13,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-31 09:53:13,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-31 09:53:13,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-31 09:53:13,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-31 09:53:13,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-31 09:53:13,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-31 09:53:13,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-31 09:53:13,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-31 09:53:13,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-31 09:53:13,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-31 09:53:13,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-31 09:53:13,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-31 09:53:13,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-31 09:53:13,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-31 09:53:13,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-31 09:53:13,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-31 09:53:13,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-31 09:53:13,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-31 09:53:13,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-31 09:53:13,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-31 09:53:13,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-31 09:53:13,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-31 09:53:13,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-31 09:53:13,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-31 09:53:13,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-31 09:53:13,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-31 09:53:13,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-31 09:53:13,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-31 09:53:13,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-31 09:53:13,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-31 09:53:13,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-31 09:53:13,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-31 09:53:13,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-31 09:53:13,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-31 09:53:13,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-31 09:53:13,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-31 09:53:13,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-31 09:53:13,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-31 09:53:13,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-31 09:53:13,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-31 09:53:13,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-31 09:53:13,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-31 09:53:13,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-31 09:53:13,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-31 09:53:13,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-31 09:53:13,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-31 09:53:13,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-31 09:53:13,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-31 09:53:13,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-31 09:53:13,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-31 09:53:13,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-31 09:53:13,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-31 09:53:13,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-31 09:53:13,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-31 09:53:13,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-31 09:53:13,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-31 09:53:13,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-31 09:53:13,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-31 09:53:13,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-31 09:53:13,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-31 09:53:13,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-31 09:53:13,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-31 09:53:13,779 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02274250642965898); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.032456480402067424, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02274250642965898); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012083333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04575, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-31 09:53:13,779 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-07-31 09:53:13,779 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.015252741922038175, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02274250642965898); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.032456480402067424, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02274250642965898); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012083333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04575, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-07-31 09:53:14,100 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 445 ms to try 5000 different iterations.  Found a solution that moves 977 regions; Going from a computed imbalance of 0.015252741922038175 to a new imbalance of 6.446801595974634E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005447890523954445); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00753592220347252); MoveCostFunction : (multiplier=7.0, imbalance=0.10855555555555556, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005447890523954445); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003916666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-31 09:53:14,102 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv208051001:81 , srv470947714:81 , srv55095226:81 , srv587962583:82 , srv2009322043:83 , srv30924677:83 , srv388184788:83 , srv510503973:83 , srv853028547:83 , srv1128773230:84 , srv1148769110:84 , srv1174342627:84 , srv12043243:84 , srv1311571387:84 , srv1355135185:84 , srv1510846052:84 , srv1561874147:84 , srv1704950811:84 , srv1793348648:84 , srv1951955274:84 , srv267337040:84 , srv671034712:84 , srv715598173:84 , srv752056354:84 , srv76563153:84 , srv921649236:84 , srv1246686271:85 , srv1247193177:85 , srv1331114204:85 , srv1407033576:85 , srv143628053:85 , srv1653953394:85 , srv1671276607:85 , srv169750908:85 , srv1712137209:85 , srv1786596284:85 , srv1804615491:85 , srv1854450003:85 , srv1864998393:85 , srv1949166327:85 , srv2050221444:85 , srv2077540095:85 , srv207781747:85 , srv2132355115:85 , srv366048020:85 , srv36935983:85 , srv864497382:85 , srv911308893:85 , srv967803605:85 , srv1017727011:86 , srv1054903096:86 , srv1079755153:86 , srv1171135551:86 , srv1221950223:86 , srv1253281314:86 , srv1269737787:86 , srv1637748916:86 , srv1700575128:86 , srv1864844699:86 , srv1980700582:86 , srv2021804248:86 , srv2114180218:86 , srv22678136:86 , srv497608842:86 , srv517147999:86 , srv628223570:86 , srv9248328:86 , srv987590192:86 , srv1032136202:87 , srv1082618739:87 , srv120410829:87 , srv1633403704:87 , srv1655264648:87 , srv1678285690:87 , srv1713968874:87 , srv1715870700:87 , srv1978559903:87 , srv215407768:87 , srv22239237:87 , srv241279565:87 , srv641392705:87 , srv64639389:87 , srv704304235:87 , srv879299439:87 , srv92293981:87 , srv120811902:88 , srv1536972090:88 , srv1592432967:88 , srv1848521095:88 , srv20584033:88 , srv2117810007:88 , srv298348055:88 , srv41831043:88 , srv521811578:88 , srv610558355:88 , srv947429201:88 , srv955435481:88 , srv1592356482:89 , srv28493324:89 , srv1399563722:530 }
2023-07-31 09:53:14,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-31 09:53:14,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2132355115=60, srv366048020=69, srv1637748916=29, srv2021804248=52, srv1713968874=38, srv388184788=71, srv497608842=74, srv1221950223=12, srv1079755153=3, srv1536972090=24, srv22239237=62, srv2117810007=59, srv1128773230=5, srv1678285690=33, srv517147999=76, srv298348055=67, srv752056354=87, srv1054903096=2, srv28493324=66, srv30924677=68, srv1331114204=18, srv41831043=72, srv2009322043=51, srv853028547=89, srv947429201=96, srv955435481=97, srv967803605=98, srv1864998393=46, srv987590192=99, srv64639389=83, srv215407768=61, srv36935983=70, srv1253281314=15, srv1854450003=44, srv1655264648=31, srv587962583=79, srv267337040=65, srv1355135185=19, srv208051001=57, srv92293981=94, srv2077540095=55, srv1247193177=14, srv76563153=88, srv120410829=9, srv241279565=64, srv1510846052=23, srv120811902=11, srv1311571387=17, srv2050221444=53, srv1174342627=8, srv20584033=54, srv169750908=34, srv1671276607=32, srv864497382=90, srv1864844699=45, srv1171135551=7, srv1592356482=26, srv12043243=10, srv628223570=81, srv704304235=85, srv1786596284=40, srv521811578=77, srv911308893=92, srv143628053=22, srv1017727011=0, srv879299439=91, srv22678136=63, srv1978559903=49, srv1592432967=27, srv55095226=78, srv9248328=95, srv1407033576=21, srv1848521095=43, srv1633403704=28, srv1653953394=30, srv641392705=82, srv1712137209=37, srv1980700582=50, srv1082618739=4, srv1246686271=13, srv1269737787=16, srv1804615491=42, srv1704950811=36, srv1399563722=20, srv510503973=75, srv1032136202=1, srv1561874147=25, srv610558355=80, srv671034712=84, srv1715870700=39, srv2114180218=58, srv1949166327=47, srv715598173=86, srv1148769110=6, srv1951955274=48, srv470947714=73, srv1700575128=35, srv1793348648=41, srv207781747=56, srv921649236=93} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-31 09:53:14,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-31 09:53:14,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-31 09:53:14,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-31 09:53:14,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-31 09:53:14,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-31 09:53:14,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-31 09:53:14,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-31 09:53:14,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-31 09:53:14,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-31 09:53:14,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-31 09:53:14,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-31 09:53:14,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-31 09:53:14,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-31 09:53:14,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-31 09:53:14,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-31 09:53:14,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-31 09:53:14,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-31 09:53:14,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-31 09:53:14,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-31 09:53:14,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-31 09:53:14,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-31 09:53:14,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-31 09:53:14,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-31 09:53:14,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-31 09:53:14,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-31 09:53:14,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-31 09:53:14,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-31 09:53:14,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-31 09:53:14,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-31 09:53:14,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-31 09:53:14,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-31 09:53:14,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-31 09:53:14,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-31 09:53:14,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-31 09:53:14,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-31 09:53:14,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-31 09:53:14,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-31 09:53:14,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-31 09:53:14,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-31 09:53:14,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-31 09:53:14,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-31 09:53:14,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-31 09:53:14,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-31 09:53:14,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-31 09:53:14,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-31 09:53:14,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-31 09:53:14,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-31 09:53:14,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-31 09:53:14,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-31 09:53:14,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-31 09:53:14,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-31 09:53:14,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-31 09:53:14,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-31 09:53:14,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-31 09:53:14,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-31 09:53:14,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-31 09:53:14,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-31 09:53:14,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-31 09:53:14,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-31 09:53:14,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-31 09:53:14,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-31 09:53:14,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-31 09:53:14,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-31 09:53:14,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-31 09:53:14,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-31 09:53:14,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-31 09:53:14,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-31 09:53:14,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-31 09:53:14,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-31 09:53:14,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-31 09:53:14,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-31 09:53:14,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-31 09:53:14,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-31 09:53:14,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-31 09:53:14,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-31 09:53:14,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-31 09:53:14,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-31 09:53:14,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-31 09:53:14,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-31 09:53:14,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-31 09:53:14,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-31 09:53:14,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-31 09:53:14,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-31 09:53:14,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-31 09:53:14,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-31 09:53:14,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-31 09:53:14,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-31 09:53:14,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-31 09:53:14,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-31 09:53:14,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-31 09:53:14,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-31 09:53:14,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-31 09:53:14,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-31 09:53:14,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-31 09:53:14,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-31 09:53:14,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-31 09:53:14,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-31 09:53:14,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-31 09:53:14,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-31 09:53:14,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-31 09:53:14,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-31 09:53:14,295 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005447890523954445); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00753592220347252); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005447890523954445); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003916666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-31 09:53:14,295 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-07-31 09:53:14,297 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=6.378390593796965E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005447890523954445); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00753592220347252); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005447890523954445); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003916666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-07-31 09:53:14,657 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 497 ms to try 5000 different iterations.  Found a solution that moves 223 regions; Going from a computed imbalance of 6.378390593796965E-4 to a new imbalance of 1.1167847645779029E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=2.2334173672426932E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.024777777777777777); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=2.2334173672426932E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-31 09:53:14,657 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv28493324:81 , srv1032136202:83 , srv1592356482:83 , srv2021804248:83 , srv41831043:83 , srv470947714:83 , srv1079755153:84 , srv1221950223:84 , srv1246686271:84 , srv1247193177:84 , srv1407033576:84 , srv143628053:84 , srv1536972090:84 , srv1561874147:84 , srv1637748916:84 , srv1653953394:84 , srv1671276607:84 , srv1700575128:84 , srv1704950811:84 , srv1713968874:84 , srv1715870700:84 , srv1786596284:84 , srv1848521095:84 , srv1864998393:84 , srv1951955274:84 , srv1978559903:84 , srv2009322043:84 , srv2050221444:84 , srv20584033:84 , srv2117810007:84 , srv22239237:84 , srv241279565:84 , srv298348055:84 , srv497608842:84 , srv517147999:84 , srv521811578:84 , srv587962583:84 , srv610558355:84 , srv628223570:84 , srv641392705:84 , srv64639389:84 , srv715598173:84 , srv92293981:84 , srv947429201:84 , srv955435481:84 , srv987590192:84 , srv1017727011:85 , srv1054903096:85 , srv1082618739:85 , srv1128773230:85 , srv1148769110:85 , srv1171135551:85 , srv1174342627:85 , srv120410829:85 , srv12043243:85 , srv120811902:85 , srv1253281314:85 , srv1269737787:85 , srv1311571387:85 , srv1331114204:85 , srv1355135185:85 , srv1592432967:85 , srv1633403704:85 , srv1655264648:85 , srv1678285690:85 , srv169750908:85 , srv1712137209:85 , srv1793348648:85 , srv1804615491:85 , srv1864844699:85 , srv1949166327:85 , srv1980700582:85 , srv2077540095:85 , srv207781747:85 , srv208051001:85 , srv2114180218:85 , srv2132355115:85 , srv215407768:85 , srv22678136:85 , srv267337040:85 , srv30924677:85 , srv36935983:85 , srv388184788:85 , srv510503973:85 , srv55095226:85 , srv671034712:85 , srv704304235:85 , srv752056354:85 , srv76563153:85 , srv853028547:85 , srv864497382:85 , srv879299439:85 , srv911308893:85 , srv921649236:85 , srv967803605:85 , srv1510846052:86 , srv1854450003:86 , srv366048020:86 , srv9248328:86 , srv1399563722:635 }
2023-07-31 09:53:14,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-31 09:53:14,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2132355115=60, srv366048020=69, srv1637748916=29, srv2021804248=52, srv1713968874=38, srv388184788=71, srv497608842=74, srv1221950223=12, srv1079755153=3, srv1536972090=24, srv22239237=62, srv2117810007=59, srv1128773230=5, srv1678285690=33, srv517147999=76, srv298348055=67, srv752056354=87, srv1054903096=2, srv28493324=66, srv30924677=68, srv1331114204=18, srv41831043=72, srv2009322043=51, srv853028547=89, srv947429201=96, srv955435481=97, srv967803605=98, srv1864998393=46, srv987590192=99, srv64639389=83, srv215407768=61, srv36935983=70, srv1253281314=15, srv1854450003=44, srv1655264648=31, srv587962583=79, srv267337040=65, srv1355135185=19, srv208051001=57, srv92293981=94, srv2077540095=55, srv1247193177=14, srv76563153=88, srv120410829=9, srv241279565=64, srv1510846052=23, srv120811902=11, srv1311571387=17, srv2050221444=53, srv1174342627=8, srv20584033=54, srv169750908=34, srv1671276607=32, srv864497382=90, srv1864844699=45, srv1171135551=7, srv1592356482=26, srv12043243=10, srv628223570=81, srv704304235=85, srv1786596284=40, srv521811578=77, srv911308893=92, srv143628053=22, srv1017727011=0, srv879299439=91, srv22678136=63, srv1978559903=49, srv1592432967=27, srv55095226=78, srv9248328=95, srv1407033576=21, srv1848521095=43, srv1633403704=28, srv1653953394=30, srv641392705=82, srv1712137209=37, srv1980700582=50, srv1082618739=4, srv1246686271=13, srv1269737787=16, srv1804615491=42, srv1704950811=36, srv1399563722=20, srv510503973=75, srv1032136202=1, srv1561874147=25, srv610558355=80, srv671034712=84, srv1715870700=39, srv2114180218=58, srv1949166327=47, srv715598173=86, srv1148769110=6, srv1951955274=48, srv470947714=73, srv1700575128=35, srv1793348648=41, srv207781747=56, srv921649236=93} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-31 09:53:14,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-31 09:53:14,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-31 09:53:14,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-31 09:53:14,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-31 09:53:14,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-31 09:53:14,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-31 09:53:14,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-31 09:53:14,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-31 09:53:14,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-31 09:53:14,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-31 09:53:14,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-31 09:53:14,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-31 09:53:14,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-31 09:53:14,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-31 09:53:14,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-31 09:53:14,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-31 09:53:14,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-31 09:53:14,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-31 09:53:14,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-31 09:53:14,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-31 09:53:14,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-31 09:53:14,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-31 09:53:14,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-31 09:53:14,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-31 09:53:14,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-31 09:53:14,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-31 09:53:14,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-31 09:53:14,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-31 09:53:14,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-31 09:53:14,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-31 09:53:14,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-31 09:53:14,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-31 09:53:14,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-31 09:53:14,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-31 09:53:14,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-31 09:53:14,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-31 09:53:14,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-31 09:53:14,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-31 09:53:14,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-31 09:53:14,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-31 09:53:14,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-31 09:53:14,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-31 09:53:14,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-31 09:53:14,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-31 09:53:14,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-31 09:53:14,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-31 09:53:14,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-31 09:53:14,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-31 09:53:14,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-31 09:53:14,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-31 09:53:14,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-31 09:53:14,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-31 09:53:14,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-31 09:53:14,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-31 09:53:14,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-31 09:53:14,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-31 09:53:14,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-31 09:53:14,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-31 09:53:14,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-31 09:53:14,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-31 09:53:14,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-31 09:53:14,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-31 09:53:14,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-31 09:53:14,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-31 09:53:14,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-31 09:53:14,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-31 09:53:14,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-31 09:53:14,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-31 09:53:14,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-31 09:53:14,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-31 09:53:14,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-31 09:53:14,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-31 09:53:14,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-31 09:53:14,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-31 09:53:14,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-31 09:53:14,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-31 09:53:14,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-31 09:53:14,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-31 09:53:14,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-31 09:53:14,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-31 09:53:14,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-31 09:53:14,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-31 09:53:14,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-31 09:53:14,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-31 09:53:14,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-31 09:53:14,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-31 09:53:14,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-31 09:53:14,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-31 09:53:14,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-31 09:53:14,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-31 09:53:14,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-31 09:53:14,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-31 09:53:14,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-31 09:53:14,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-31 09:53:14,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-31 09:53:14,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-31 09:53:14,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-31 09:53:14,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-31 09:53:14,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-31 09:53:14,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-31 09:53:14,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-31 09:53:14,807 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=9.606368271611158E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=2.2334173672426932E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=2.2334173672426932E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-31 09:53:14,808 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv28493324:81 , srv1032136202:83 , srv1592356482:83 , srv2021804248:83 , srv41831043:83 , srv470947714:83 , srv1079755153:84 , srv1221950223:84 , srv1246686271:84 , srv1247193177:84 , srv1407033576:84 , srv143628053:84 , srv1536972090:84 , srv1561874147:84 , srv1637748916:84 , srv1653953394:84 , srv1671276607:84 , srv1700575128:84 , srv1704950811:84 , srv1713968874:84 , srv1715870700:84 , srv1786596284:84 , srv1848521095:84 , srv1864998393:84 , srv1951955274:84 , srv1978559903:84 , srv2009322043:84 , srv2050221444:84 , srv20584033:84 , srv2117810007:84 , srv22239237:84 , srv241279565:84 , srv298348055:84 , srv497608842:84 , srv517147999:84 , srv521811578:84 , srv587962583:84 , srv610558355:84 , srv628223570:84 , srv641392705:84 , srv64639389:84 , srv715598173:84 , srv92293981:84 , srv947429201:84 , srv955435481:84 , srv987590192:84 , srv1017727011:85 , srv1054903096:85 , srv1082618739:85 , srv1128773230:85 , srv1148769110:85 , srv1171135551:85 , srv1174342627:85 , srv120410829:85 , srv12043243:85 , srv120811902:85 , srv1253281314:85 , srv1269737787:85 , srv1311571387:85 , srv1331114204:85 , srv1355135185:85 , srv1592432967:85 , srv1633403704:85 , srv1655264648:85 , srv1678285690:85 , srv169750908:85 , srv1712137209:85 , srv1793348648:85 , srv1804615491:85 , srv1864844699:85 , srv1949166327:85 , srv1980700582:85 , srv2077540095:85 , srv207781747:85 , srv208051001:85 , srv2114180218:85 , srv2132355115:85 , srv215407768:85 , srv22678136:85 , srv267337040:85 , srv30924677:85 , srv36935983:85 , srv388184788:85 , srv510503973:85 , srv55095226:85 , srv671034712:85 , srv704304235:85 , srv752056354:85 , srv76563153:85 , srv853028547:85 , srv864497382:85 , srv879299439:85 , srv911308893:85 , srv921649236:85 , srv967803605:85 , srv1510846052:86 , srv1854450003:86 , srv366048020:86 , srv9248328:86 , srv1399563722:635 }
2023-07-31 09:53:14,858 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=413 (was 345) - SystemLoadAverage LEAK? -, ProcessCount=174 (was 174), AvailableMemoryMB=1669 (was 2487)
2023-07-31 09:53:14,868 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=413, ProcessCount=174, AvailableMemoryMB=1668
2023-07-31 09:53:14,869 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-07-31 09:53:14,870 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-07-31 09:53:14,900 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1008513473:84 , srv100969623:84 , srv1058228396:84 , srv1087248783:84 , srv1156549324:84 , srv1442119244:84 , srv1569530896:84 , srv1782864281:84 , srv1873869886:84 , srv1889961599:84 , srv1934064021:84 , srv1960791024:84 , srv1962646883:84 , srv1987822462:84 , srv2032355834:84 , srv2080235306:84 , srv271508140:84 , srv289852926:84 , srv44082021:84 , srv45014112:84 , srv467765417:84 , srv474102552:84 , srv486343267:84 , srv506808822:84 , srv61771625:84 , srv81253386:84 , srv876325898:84 , srv926477739:84 , srv957748580:84 , srv1647421657:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-07-31 09:53:14,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-31 09:53:14,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv45014112=20, srv1889961599=10, srv1647421657=7, srv100969623=1, srv1934064021=11, srv467765417=21, srv926477739=28, srv1156549324=4, srv1569530896=6, srv1087248783=3, srv1962646883=13, srv2032355834=15, srv289852926=18, srv474102552=22, srv1987822462=14, srv506808822=24, srv1873869886=9, srv1008513473=0, srv1442119244=5, srv2080235306=16, srv876325898=27, srv44082021=19, srv486343267=23, srv1960791024=12, srv1782864281=8, srv957748580=29, srv81253386=26, srv271508140=17, srv1058228396=2, srv61771625=25} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-31 09:53:14,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-31 09:53:14,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-31 09:53:14,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-31 09:53:14,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-31 09:53:14,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-31 09:53:14,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-31 09:53:14,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-31 09:53:14,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-31 09:53:14,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-31 09:53:14,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-31 09:53:14,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-31 09:53:14,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-31 09:53:14,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-31 09:53:14,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-31 09:53:14,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-31 09:53:14,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-31 09:53:14,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-31 09:53:14,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-31 09:53:14,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-31 09:53:14,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-31 09:53:14,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-31 09:53:14,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-31 09:53:14,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-31 09:53:14,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-31 09:53:14,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-31 09:53:14,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-31 09:53:14,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-31 09:53:14,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-31 09:53:14,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-31 09:53:14,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-31 09:53:14,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-07-31 09:53:14,949 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0801873133780715, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-31 09:53:14,951 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912445710870378, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0801873133780715, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-07-31 09:55:14,931 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5373398 different iterations.  Found a solution that moves 1870 regions; Going from a computed imbalance of 0.9912445710870378 to a new imbalance of 4.3646732880327594E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6925925925925925, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-31 09:55:14,937 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1008513473:90 , srv100969623:90 , srv1058228396:90 , srv1087248783:90 , srv1156549324:90 , srv1442119244:90 , srv1569530896:90 , srv1647421657:90 , srv1782864281:90 , srv1873869886:90 , srv1889961599:90 , srv1934064021:90 , srv1960791024:90 , srv1962646883:90 , srv1987822462:90 , srv2032355834:90 , srv2080235306:90 , srv271508140:90 , srv289852926:90 , srv44082021:90 , srv45014112:90 , srv467765417:90 , srv474102552:90 , srv486343267:90 , srv506808822:90 , srv61771625:90 , srv81253386:90 , srv876325898:90 , srv926477739:90 , srv957748580:90 }
2023-07-31 09:55:14,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-31 09:55:14,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv45014112=20, srv1889961599=10, srv1647421657=7, srv100969623=1, srv1934064021=11, srv467765417=21, srv926477739=28, srv1156549324=4, srv1569530896=6, srv1087248783=3, srv1962646883=13, srv2032355834=15, srv289852926=18, srv474102552=22, srv1987822462=14, srv506808822=24, srv1873869886=9, srv1008513473=0, srv1442119244=5, srv2080235306=16, srv876325898=27, srv44082021=19, srv486343267=23, srv1960791024=12, srv1782864281=8, srv957748580=29, srv81253386=26, srv271508140=17, srv1058228396=2, srv61771625=25} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-31 09:55:14,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-31 09:55:14,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-31 09:55:14,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-31 09:55:14,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-31 09:55:14,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-31 09:55:14,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-31 09:55:14,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-31 09:55:14,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-31 09:55:14,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-31 09:55:14,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-31 09:55:14,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-31 09:55:14,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-31 09:55:14,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-31 09:55:14,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-31 09:55:14,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-31 09:55:14,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-31 09:55:14,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-31 09:55:14,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-31 09:55:14,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-31 09:55:14,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-31 09:55:14,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-31 09:55:14,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-31 09:55:14,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-31 09:55:14,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-31 09:55:14,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-31 09:55:14,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-31 09:55:14,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-31 09:55:14,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-31 09:55:14,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-31 09:55:14,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-31 09:55:14,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-07-31 09:55:14,954 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-31 09:55:14,954 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1008513473:90 , srv100969623:90 , srv1058228396:90 , srv1087248783:90 , srv1156549324:90 , srv1442119244:90 , srv1569530896:90 , srv1647421657:90 , srv1782864281:90 , srv1873869886:90 , srv1889961599:90 , srv1934064021:90 , srv1960791024:90 , srv1962646883:90 , srv1987822462:90 , srv2032355834:90 , srv2080235306:90 , srv271508140:90 , srv289852926:90 , srv44082021:90 , srv45014112:90 , srv467765417:90 , srv474102552:90 , srv486343267:90 , srv506808822:90 , srv61771625:90 , srv81253386:90 , srv876325898:90 , srv926477739:90 , srv957748580:90 }
2023-07-31 09:55:14,970 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=196 (was 413), ProcessCount=172 (was 174), AvailableMemoryMB=1449 (was 1668)