2023-10-03 18:53:25,482 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-10-03 18:53:25,760 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-10-03 18:53:25,864 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-10-03 18:53:25,929 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-10-03 18:53:25,939 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-10-03 18:53:25,953 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=297, ProcessCount=170, AvailableMemoryMB=8759
2023-10-03 18:53:25,955 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-10-03 18:53:25,956 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-10-03 18:53:26,101 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1000439481:84 , srv1027430101:84 , srv1028308513:84 , srv1065202414:84 , srv1079986837:84 , srv1118383304:84 , srv1123624951:84 , srv1130093103:84 , srv1148351270:84 , srv1154941282:84 , srv1164112296:84 , srv119621826:84 , srv1200718078:84 , srv1204478931:84 , srv1212721765:84 , srv1264296761:84 , srv1267551397:84 , srv1319356539:84 , srv1320391385:84 , srv1341874019:84 , srv1346507235:84 , srv1367253232:84 , srv1413106800:84 , srv1438246720:84 , srv1458135232:84 , srv1462156661:84 , srv1472458599:84 , srv1517171282:84 , srv1570635312:84 , srv1585168174:84 , srv1604433647:84 , srv1620710337:84 , srv1622232158:84 , srv1633176335:84 , srv1675112244:84 , srv1687574320:84 , srv1716854338:84 , srv1717151055:84 , srv174742690:84 , srv178331828:84 , srv1796113219:84 , srv1797720940:84 , srv1808794511:84 , srv1819772915:84 , srv1821188964:84 , srv1829098249:84 , srv1840734068:84 , srv1841590705:84 , srv1850626024:84 , srv1864917661:84 , srv1970140163:84 , srv2002687294:84 , srv2051399124:84 , srv2061052211:84 , srv2066923478:84 , srv2116427915:84 , srv2120039846:84 , srv2143753856:84 , srv2145053026:84 , srv218590600:84 , srv272197286:84 , srv278409005:84 , srv332755048:84 , srv337208024:84 , srv337783787:84 , srv371046896:84 , srv373842725:84 , srv378751723:84 , srv39253393:84 , srv403781194:84 , srv416221137:84 , srv433786686:84 , srv46009038:84 , srv469258617:84 , srv488156383:84 , srv502737949:84 , srv540037980:84 , srv564652720:84 , srv577670361:84 , srv57806747:84 , srv605003136:84 , srv626144805:84 , srv640409694:84 , srv651690088:84 , srv67904614:84 , srv693656483:84 , srv693834011:84 , srv698277412:84 , srv710464330:84 , srv785437484:84 , srv802117795:84 , srv810510719:84 , srv856589825:84 , srv887347300:84 , srv889712869:84 , srv901473027:84 , srv905241:84 , srv912414079:84 , srv942742178:84 , srv1581297441:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-10-03 18:53:26,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-03 18:53:26,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv278409005=62, srv1716854338=37, srv651690088=84, srv1154941282=9, srv178331828=40, srv1581297441=29, srv905241=97, srv2143753856=58, srv1123624951=6, srv540037980=77, srv488156383=75, srv1829098249=46, srv1164112296=10, srv693656483=86, srv901473027=96, srv1319356539=17, srv1620710337=32, srv1864917661=50, srv785437484=90, srv1264296761=15, srv2145053026=59, srv337783787=65, srv46009038=73, srv1367253232=21, srv67904614=85, srv57806747=80, srv1458135232=24, srv1970140163=51, srv2051399124=53, srv2120039846=57, srv1148351270=8, srv119621826=11, srv640409694=83, srv698277412=88, srv174742690=39, srv889712869=95, srv1320391385=18, srv39253393=69, srv2066923478=55, srv1717151055=38, srv1413106800=22, srv1622232158=33, srv469258617=74, srv1346507235=20, srv337208024=64, srv416221137=71, srv856589825=93, srv626144805=82, srv1065202414=3, srv912414079=98, srv1267551397=16, srv1687574320=36, srv1633176335=34, srv1796113219=41, srv1079986837=4, srv1472458599=26, srv371046896=66, srv332755048=63, srv802117795=91, srv1819772915=44, srv887347300=94, srv1118383304=5, srv433786686=72, srv693834011=87, srv1341874019=19, srv378751723=68, srv1028308513=2, srv1808794511=43, srv1850626024=49, srv1821188964=45, srv1027430101=1, srv1570635312=28, srv1675112244=35, srv564652720=78, srv2116427915=56, srv218590600=60, srv1200718078=12, srv605003136=81, srv1204478931=13, srv1438246720=23, srv710464330=89, srv1462156661=25, srv2061052211=54, srv373842725=67, srv942742178=99, srv1517171282=27, srv1840734068=47, srv1000439481=0, srv1212721765=14, srv1841590705=48, srv1130093103=7, srv1585168174=30, srv1797720940=42, srv272197286=61, srv577670361=79, srv1604433647=31, srv2002687294=52, srv810510719=92, srv403781194=70, srv502737949=76} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-03 18:53:26,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-03 18:53:26,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-03 18:53:26,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-03 18:53:26,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-03 18:53:26,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-03 18:53:26,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-03 18:53:26,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-03 18:53:26,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-03 18:53:26,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-03 18:53:26,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-03 18:53:26,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-03 18:53:26,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-03 18:53:26,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-03 18:53:26,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-03 18:53:26,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-03 18:53:26,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-03 18:53:26,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-03 18:53:26,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-03 18:53:26,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-03 18:53:26,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-03 18:53:26,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-03 18:53:26,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-03 18:53:26,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-03 18:53:26,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-03 18:53:26,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-03 18:53:26,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-03 18:53:26,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-03 18:53:26,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-03 18:53:26,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-03 18:53:26,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-03 18:53:26,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-03 18:53:26,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-03 18:53:26,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-03 18:53:26,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-03 18:53:26,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-03 18:53:26,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-03 18:53:26,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-03 18:53:26,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-03 18:53:26,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-03 18:53:26,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-03 18:53:26,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-03 18:53:26,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-03 18:53:26,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-03 18:53:26,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-03 18:53:26,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-03 18:53:26,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-03 18:53:26,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-03 18:53:26,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-03 18:53:26,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-03 18:53:26,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-03 18:53:26,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-03 18:53:26,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-03 18:53:26,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-03 18:53:26,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-03 18:53:26,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-03 18:53:26,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-03 18:53:26,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-03 18:53:26,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-03 18:53:26,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-03 18:53:26,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-03 18:53:26,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-03 18:53:26,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-03 18:53:26,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-03 18:53:26,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-03 18:53:26,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-03 18:53:26,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-03 18:53:26,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-03 18:53:26,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-03 18:53:26,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-03 18:53:26,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-03 18:53:26,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-03 18:53:26,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-03 18:53:26,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-03 18:53:26,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-03 18:53:26,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-03 18:53:26,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-03 18:53:26,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-03 18:53:26,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-03 18:53:26,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-03 18:53:26,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-03 18:53:26,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-03 18:53:26,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-03 18:53:26,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-03 18:53:26,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-03 18:53:26,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-03 18:53:26,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-03 18:53:26,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-03 18:53:26,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-03 18:53:26,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-03 18:53:26,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-03 18:53:26,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-03 18:53:26,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-03 18:53:26,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-03 18:53:26,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-03 18:53:26,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-03 18:53:26,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-03 18:53:26,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-03 18:53:26,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-03 18:53:26,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-03 18:53:26,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-03 18:53:26,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-03 18:53:26,521 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-03 18:53:26,521 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-03 18:53:26,522 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-03 18:53:27,278 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 1067 ms to try 5000 different iterations.  Found a solution that moves 4689 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18085932873690852. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0413197303322217, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.053783871430366184, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.521, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0413197303322217, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.176, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24366666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-03 18:53:27,319 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1346507235:80 , srv1585168174:81 , srv2116427915:81 , srv810510719:81 , srv942742178:81 , srv1148351270:82 , srv1212721765:82 , srv1797720940:82 , srv2061052211:82 , srv373842725:82 , srv502737949:82 , srv640409694:82 , srv1154941282:83 , srv1200718078:83 , srv1462156661:83 , srv1717151055:83 , srv1808794511:83 , srv332755048:83 , srv433786686:83 , srv1267551397:84 , srv1620710337:84 , srv1970140163:84 , srv2051399124:84 , srv337783787:84 , srv378751723:84 , srv403781194:84 , srv912414079:84 , srv1458135232:85 , srv1517171282:85 , srv1570635312:85 , srv1821188964:85 , srv2120039846:85 , srv416221137:85 , srv57806747:85 , srv698277412:85 , srv887347300:85 , srv889712869:85 , srv1164112296:86 , srv1320391385:86 , srv1472458599:86 , srv174742690:86 , srv1796113219:86 , srv1819772915:86 , srv1850626024:86 , srv2143753856:86 , srv39253393:86 , srv46009038:86 , srv564652720:86 , srv626144805:86 , srv785437484:86 , srv1027430101:87 , srv1028308513:87 , srv1079986837:87 , srv119621826:87 , srv1264296761:87 , srv1319356539:87 , srv1413106800:87 , srv1438246720:87 , srv1622232158:87 , srv178331828:87 , srv1829098249:87 , srv1864917661:87 , srv2145053026:87 , srv218590600:87 , srv278409005:87 , srv577670361:87 , srv605003136:87 , srv693834011:87 , srv802117795:87 , srv1065202414:88 , srv1118383304:88 , srv1204478931:88 , srv1604433647:88 , srv1633176335:88 , srv1840734068:88 , srv2002687294:88 , srv337208024:88 , srv488156383:88 , srv901473027:88 , srv1123624951:89 , srv1367253232:89 , srv1716854338:89 , srv1841590705:89 , srv2066923478:89 , srv272197286:89 , srv651690088:89 , srv1130093103:90 , srv371046896:90 , srv856589825:90 , srv1000439481:91 , srv540037980:91 , srv67904614:91 , srv693656483:91 , srv905241:91 , srv1341874019:92 , srv1687574320:92 , srv710464330:92 , srv469258617:93 , srv1675112244:94 , srv1581297441:457 }
2023-10-03 18:53:27,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-03 18:53:27,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv278409005=62, srv1716854338=37, srv651690088=84, srv1154941282=9, srv178331828=40, srv1581297441=29, srv905241=97, srv2143753856=58, srv1123624951=6, srv540037980=77, srv488156383=75, srv1829098249=46, srv1164112296=10, srv693656483=86, srv901473027=96, srv1319356539=17, srv1620710337=32, srv1864917661=50, srv785437484=90, srv1264296761=15, srv2145053026=59, srv337783787=65, srv46009038=73, srv1367253232=21, srv67904614=85, srv57806747=80, srv1458135232=24, srv1970140163=51, srv2051399124=53, srv2120039846=57, srv1148351270=8, srv119621826=11, srv640409694=83, srv698277412=88, srv174742690=39, srv889712869=95, srv1320391385=18, srv39253393=69, srv2066923478=55, srv1717151055=38, srv1413106800=22, srv1622232158=33, srv469258617=74, srv1346507235=20, srv337208024=64, srv416221137=71, srv856589825=93, srv626144805=82, srv1065202414=3, srv912414079=98, srv1267551397=16, srv1687574320=36, srv1633176335=34, srv1796113219=41, srv1079986837=4, srv1472458599=26, srv371046896=66, srv332755048=63, srv802117795=91, srv1819772915=44, srv887347300=94, srv1118383304=5, srv433786686=72, srv693834011=87, srv1341874019=19, srv378751723=68, srv1028308513=2, srv1808794511=43, srv1850626024=49, srv1821188964=45, srv1027430101=1, srv1570635312=28, srv1675112244=35, srv564652720=78, srv2116427915=56, srv218590600=60, srv1200718078=12, srv605003136=81, srv1204478931=13, srv1438246720=23, srv710464330=89, srv1462156661=25, srv2061052211=54, srv373842725=67, srv942742178=99, srv1517171282=27, srv1840734068=47, srv1000439481=0, srv1212721765=14, srv1841590705=48, srv1130093103=7, srv1585168174=30, srv1797720940=42, srv272197286=61, srv577670361=79, srv1604433647=31, srv2002687294=52, srv810510719=92, srv403781194=70, srv502737949=76} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-03 18:53:27,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-03 18:53:27,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-03 18:53:27,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-03 18:53:27,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-03 18:53:27,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-03 18:53:27,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-03 18:53:27,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-03 18:53:27,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-03 18:53:27,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-03 18:53:27,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-03 18:53:27,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-03 18:53:27,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-03 18:53:27,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-03 18:53:27,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-03 18:53:27,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-03 18:53:27,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-03 18:53:27,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-03 18:53:27,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-03 18:53:27,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-03 18:53:27,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-03 18:53:27,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-03 18:53:27,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-03 18:53:27,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-03 18:53:27,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-03 18:53:27,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-03 18:53:27,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-03 18:53:27,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-03 18:53:27,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-03 18:53:27,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-03 18:53:27,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-03 18:53:27,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-03 18:53:27,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-03 18:53:27,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-03 18:53:27,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-03 18:53:27,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-03 18:53:27,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-03 18:53:27,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-03 18:53:27,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-03 18:53:27,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-03 18:53:27,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-03 18:53:27,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-03 18:53:27,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-03 18:53:27,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-03 18:53:27,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-03 18:53:27,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-03 18:53:27,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-03 18:53:27,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-03 18:53:27,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-03 18:53:27,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-03 18:53:27,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-03 18:53:27,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-03 18:53:27,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-03 18:53:27,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-03 18:53:27,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-03 18:53:27,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-03 18:53:27,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-03 18:53:27,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-03 18:53:27,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-03 18:53:27,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-03 18:53:27,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-03 18:53:27,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-03 18:53:27,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-03 18:53:27,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-03 18:53:27,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-03 18:53:27,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-03 18:53:27,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-03 18:53:27,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-03 18:53:27,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-03 18:53:27,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-03 18:53:27,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-03 18:53:27,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-03 18:53:27,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-03 18:53:27,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-03 18:53:27,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-03 18:53:27,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-03 18:53:27,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-03 18:53:27,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-03 18:53:27,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-03 18:53:27,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-03 18:53:27,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-03 18:53:27,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-03 18:53:27,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-03 18:53:27,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-03 18:53:27,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-03 18:53:27,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-03 18:53:27,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-03 18:53:27,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-03 18:53:27,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-03 18:53:27,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-03 18:53:27,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-03 18:53:27,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-03 18:53:27,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-03 18:53:27,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-03 18:53:27,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-03 18:53:27,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-03 18:53:27,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-03 18:53:27,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-03 18:53:27,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-03 18:53:27,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-03 18:53:27,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-03 18:53:27,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-03 18:53:27,524 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0413197303322217, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.053783871430366184, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0413197303322217, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.176, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24366666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-03 18:53:27,524 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-03 18:53:27,524 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18082649565715303, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0413197303322217, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.053783871430366184, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0413197303322217, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.176, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24366666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-03 18:53:27,830 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 434 ms to try 5000 different iterations.  Found a solution that moves 2643 regions; Going from a computed imbalance of 0.18082649565715303 to a new imbalance of 0.014561628386027262. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022384369156656255); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03352805080521171, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2936666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022384369156656255); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011583333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.042833333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-03 18:53:27,842 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1000439481:80 , srv469258617:80 , srv905241:80 , srv1341874019:81 , srv1687574320:81 , srv1841590705:81 , srv371046896:81 , srv540037980:81 , srv1130093103:82 , srv1675112244:82 , srv1864917661:82 , srv67904614:82 , srv693656483:82 , srv710464330:82 , srv1319356539:83 , srv1367253232:83 , srv1633176335:83 , srv1716854338:83 , srv2066923478:83 , srv272197286:83 , srv605003136:83 , srv651690088:83 , srv856589825:83 , srv1028308513:84 , srv1118383304:84 , srv1123624951:84 , srv1164112296:84 , srv1204478931:84 , srv1438246720:84 , srv178331828:84 , srv1840734068:84 , srv2002687294:84 , srv378751723:84 , srv39253393:84 , srv577670361:84 , srv802117795:84 , srv1079986837:85 , srv119621826:85 , srv1264296761:85 , srv1517171282:85 , srv1604433647:85 , srv174742690:85 , srv1829098249:85 , srv1850626024:85 , srv2120039846:85 , srv2145053026:85 , srv337208024:85 , srv901473027:85 , srv1027430101:86 , srv1065202414:86 , srv1413106800:86 , srv1458135232:86 , srv1472458599:86 , srv1622232158:86 , srv1819772915:86 , srv2051399124:86 , srv2143753856:86 , srv278409005:86 , srv416221137:86 , srv488156383:86 , srv626144805:86 , srv785437484:86 , srv1267551397:87 , srv1320391385:87 , srv1796113219:87 , srv1821188964:87 , srv218590600:87 , srv46009038:87 , srv698277412:87 , srv889712869:87 , srv1212721765:88 , srv1570635312:88 , srv1717151055:88 , srv1970140163:88 , srv2116427915:88 , srv337783787:88 , srv433786686:88 , srv564652720:88 , srv57806747:88 , srv693834011:88 , srv912414079:88 , srv1620710337:89 , srv1797720940:89 , srv2061052211:89 , srv332755048:89 , srv403781194:89 , srv502737949:89 , srv1148351270:90 , srv1200718078:90 , srv1462156661:90 , srv1808794511:90 , srv373842725:90 , srv640409694:90 , srv1154941282:91 , srv1346507235:91 , srv1585168174:91 , srv810510719:91 , srv887347300:91 , srv942742178:91 , srv1581297441:516 }
2023-10-03 18:53:27,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-03 18:53:27,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv278409005=62, srv1716854338=37, srv651690088=84, srv1154941282=9, srv178331828=40, srv1581297441=29, srv905241=97, srv2143753856=58, srv1123624951=6, srv540037980=77, srv488156383=75, srv1829098249=46, srv1164112296=10, srv693656483=86, srv901473027=96, srv1319356539=17, srv1620710337=32, srv1864917661=50, srv785437484=90, srv1264296761=15, srv2145053026=59, srv337783787=65, srv46009038=73, srv1367253232=21, srv67904614=85, srv57806747=80, srv1458135232=24, srv1970140163=51, srv2051399124=53, srv2120039846=57, srv1148351270=8, srv119621826=11, srv640409694=83, srv698277412=88, srv174742690=39, srv889712869=95, srv1320391385=18, srv39253393=69, srv2066923478=55, srv1717151055=38, srv1413106800=22, srv1622232158=33, srv469258617=74, srv1346507235=20, srv337208024=64, srv416221137=71, srv856589825=93, srv626144805=82, srv1065202414=3, srv912414079=98, srv1267551397=16, srv1687574320=36, srv1633176335=34, srv1796113219=41, srv1079986837=4, srv1472458599=26, srv371046896=66, srv332755048=63, srv802117795=91, srv1819772915=44, srv887347300=94, srv1118383304=5, srv433786686=72, srv693834011=87, srv1341874019=19, srv378751723=68, srv1028308513=2, srv1808794511=43, srv1850626024=49, srv1821188964=45, srv1027430101=1, srv1570635312=28, srv1675112244=35, srv564652720=78, srv2116427915=56, srv218590600=60, srv1200718078=12, srv605003136=81, srv1204478931=13, srv1438246720=23, srv710464330=89, srv1462156661=25, srv2061052211=54, srv373842725=67, srv942742178=99, srv1517171282=27, srv1840734068=47, srv1000439481=0, srv1212721765=14, srv1841590705=48, srv1130093103=7, srv1585168174=30, srv1797720940=42, srv272197286=61, srv577670361=79, srv1604433647=31, srv2002687294=52, srv810510719=92, srv403781194=70, srv502737949=76} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-03 18:53:27,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-03 18:53:27,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-03 18:53:27,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-03 18:53:27,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-03 18:53:27,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-03 18:53:27,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-03 18:53:27,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-03 18:53:27,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-03 18:53:27,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-03 18:53:27,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-03 18:53:27,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-03 18:53:27,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-03 18:53:27,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-03 18:53:27,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-03 18:53:27,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-03 18:53:27,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-03 18:53:27,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-03 18:53:27,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-03 18:53:27,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-03 18:53:27,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-03 18:53:27,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-03 18:53:27,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-03 18:53:27,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-03 18:53:27,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-03 18:53:27,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-03 18:53:27,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-03 18:53:27,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-03 18:53:27,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-03 18:53:27,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-03 18:53:27,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-03 18:53:27,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-03 18:53:27,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-03 18:53:27,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-03 18:53:27,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-03 18:53:27,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-03 18:53:27,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-03 18:53:27,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-03 18:53:27,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-03 18:53:27,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-03 18:53:27,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-03 18:53:27,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-03 18:53:27,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-03 18:53:27,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-03 18:53:27,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-03 18:53:27,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-03 18:53:27,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-03 18:53:27,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-03 18:53:27,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-03 18:53:27,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-03 18:53:27,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-03 18:53:27,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-03 18:53:27,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-03 18:53:27,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-03 18:53:27,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-03 18:53:27,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-03 18:53:27,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-03 18:53:27,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-03 18:53:27,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-03 18:53:27,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-03 18:53:27,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-03 18:53:27,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-03 18:53:27,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-03 18:53:27,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-03 18:53:27,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-03 18:53:27,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-03 18:53:27,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-03 18:53:27,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-03 18:53:27,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-03 18:53:27,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-03 18:53:27,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-03 18:53:27,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-03 18:53:27,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-03 18:53:27,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-03 18:53:27,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-03 18:53:27,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-03 18:53:27,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-03 18:53:27,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-03 18:53:27,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-03 18:53:27,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-03 18:53:27,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-03 18:53:27,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-03 18:53:27,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-03 18:53:27,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-03 18:53:27,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-03 18:53:27,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-03 18:53:27,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-03 18:53:27,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-03 18:53:27,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-03 18:53:27,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-03 18:53:27,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-03 18:53:27,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-03 18:53:27,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-03 18:53:27,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-03 18:53:27,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-03 18:53:27,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-03 18:53:27,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-03 18:53:27,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-03 18:53:27,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-03 18:53:27,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-03 18:53:27,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-03 18:53:27,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-03 18:53:28,029 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022384369156656255); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03352805080521171, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022384369156656255); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011583333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.042833333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-03 18:53:28,029 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-03 18:53:28,030 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.014543121704476025, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022384369156656255); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03352805080521171, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022384369156656255); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011583333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.042833333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-03 18:53:28,319 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 411 ms to try 5000 different iterations.  Found a solution that moves 994 regions; Going from a computed imbalance of 0.014543121704476025 to a new imbalance of 4.8032061466492883E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0047588039714625595); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006733670025252104); MoveCostFunction : (multiplier=7.0, imbalance=0.11044444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0047588039714625595); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-03 18:53:28,322 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv887347300:81 , srv1675112244:82 , srv1154941282:83 , srv488156383:83 , srv564652720:83 , srv693834011:83 , srv710464330:83 , srv1065202414:84 , srv1123624951:84 , srv1200718078:84 , srv1320391385:84 , srv1796113219:84 , srv1808794511:84 , srv218590600:84 , srv46009038:84 , srv57806747:84 , srv901473027:84 , srv1027430101:85 , srv1079986837:85 , srv1130093103:85 , srv1148351270:85 , srv1341874019:85 , srv1413106800:85 , srv1462156661:85 , srv1570635312:85 , srv1585168174:85 , srv1604433647:85 , srv1620710337:85 , srv1622232158:85 , srv1687574320:85 , srv1821188964:85 , srv1970140163:85 , srv2002687294:85 , srv2066923478:85 , srv272197286:85 , srv278409005:85 , srv332755048:85 , srv337208024:85 , srv337783787:85 , srv403781194:85 , srv469258617:85 , srv540037980:85 , srv626144805:85 , srv651690088:85 , srv693656483:85 , srv698277412:85 , srv810510719:85 , srv856589825:85 , srv942742178:85 , srv1118383304:86 , srv119621826:86 , srv1204478931:86 , srv1346507235:86 , srv1367253232:86 , srv1438246720:86 , srv1458135232:86 , srv1472458599:86 , srv178331828:86 , srv1819772915:86 , srv1829098249:86 , srv1840734068:86 , srv1850626024:86 , srv2061052211:86 , srv2143753856:86 , srv2145053026:86 , srv373842725:86 , srv433786686:86 , srv502737949:86 , srv640409694:86 , srv67904614:86 , srv785437484:86 , srv889712869:86 , srv905241:86 , srv1000439481:87 , srv1028308513:87 , srv1164112296:87 , srv1212721765:87 , srv1264296761:87 , srv1267551397:87 , srv1319356539:87 , srv1633176335:87 , srv1716854338:87 , srv1717151055:87 , srv174742690:87 , srv1797720940:87 , srv2051399124:87 , srv371046896:87 , srv416221137:87 , srv577670361:87 , srv605003136:87 , srv802117795:87 , srv912414079:87 , srv1517171282:88 , srv1841590705:88 , srv1864917661:88 , srv2116427915:88 , srv2120039846:88 , srv378751723:89 , srv39253393:89 , srv1581297441:527 }
2023-10-03 18:53:28,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-03 18:53:28,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv278409005=62, srv1716854338=37, srv651690088=84, srv1154941282=9, srv178331828=40, srv1581297441=29, srv905241=97, srv2143753856=58, srv1123624951=6, srv540037980=77, srv488156383=75, srv1829098249=46, srv1164112296=10, srv693656483=86, srv901473027=96, srv1319356539=17, srv1620710337=32, srv1864917661=50, srv785437484=90, srv1264296761=15, srv2145053026=59, srv337783787=65, srv46009038=73, srv1367253232=21, srv67904614=85, srv57806747=80, srv1458135232=24, srv1970140163=51, srv2051399124=53, srv2120039846=57, srv1148351270=8, srv119621826=11, srv640409694=83, srv698277412=88, srv174742690=39, srv889712869=95, srv1320391385=18, srv39253393=69, srv2066923478=55, srv1717151055=38, srv1413106800=22, srv1622232158=33, srv469258617=74, srv1346507235=20, srv337208024=64, srv416221137=71, srv856589825=93, srv626144805=82, srv1065202414=3, srv912414079=98, srv1267551397=16, srv1687574320=36, srv1633176335=34, srv1796113219=41, srv1079986837=4, srv1472458599=26, srv371046896=66, srv332755048=63, srv802117795=91, srv1819772915=44, srv887347300=94, srv1118383304=5, srv433786686=72, srv693834011=87, srv1341874019=19, srv378751723=68, srv1028308513=2, srv1808794511=43, srv1850626024=49, srv1821188964=45, srv1027430101=1, srv1570635312=28, srv1675112244=35, srv564652720=78, srv2116427915=56, srv218590600=60, srv1200718078=12, srv605003136=81, srv1204478931=13, srv1438246720=23, srv710464330=89, srv1462156661=25, srv2061052211=54, srv373842725=67, srv942742178=99, srv1517171282=27, srv1840734068=47, srv1000439481=0, srv1212721765=14, srv1841590705=48, srv1130093103=7, srv1585168174=30, srv1797720940=42, srv272197286=61, srv577670361=79, srv1604433647=31, srv2002687294=52, srv810510719=92, srv403781194=70, srv502737949=76} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-03 18:53:28,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-03 18:53:28,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-03 18:53:28,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-03 18:53:28,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-03 18:53:28,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-03 18:53:28,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-03 18:53:28,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-03 18:53:28,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-03 18:53:28,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-03 18:53:28,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-03 18:53:28,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-03 18:53:28,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-03 18:53:28,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-03 18:53:28,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-03 18:53:28,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-03 18:53:28,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-03 18:53:28,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-03 18:53:28,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-03 18:53:28,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-03 18:53:28,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-03 18:53:28,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-03 18:53:28,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-03 18:53:28,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-03 18:53:28,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-03 18:53:28,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-03 18:53:28,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-03 18:53:28,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-03 18:53:28,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-03 18:53:28,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-03 18:53:28,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-03 18:53:28,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-03 18:53:28,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-03 18:53:28,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-03 18:53:28,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-03 18:53:28,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-03 18:53:28,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-03 18:53:28,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-03 18:53:28,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-03 18:53:28,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-03 18:53:28,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-03 18:53:28,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-03 18:53:28,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-03 18:53:28,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-03 18:53:28,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-03 18:53:28,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-03 18:53:28,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-03 18:53:28,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-03 18:53:28,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-03 18:53:28,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-03 18:53:28,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-03 18:53:28,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-03 18:53:28,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-03 18:53:28,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-03 18:53:28,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-03 18:53:28,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-03 18:53:28,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-03 18:53:28,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-03 18:53:28,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-03 18:53:28,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-03 18:53:28,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-03 18:53:28,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-03 18:53:28,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-03 18:53:28,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-03 18:53:28,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-03 18:53:28,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-03 18:53:28,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-03 18:53:28,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-03 18:53:28,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-03 18:53:28,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-03 18:53:28,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-03 18:53:28,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-03 18:53:28,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-03 18:53:28,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-03 18:53:28,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-03 18:53:28,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-03 18:53:28,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-03 18:53:28,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-03 18:53:28,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-03 18:53:28,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-03 18:53:28,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-03 18:53:28,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-03 18:53:28,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-03 18:53:28,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-03 18:53:28,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-03 18:53:28,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-03 18:53:28,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-03 18:53:28,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-03 18:53:28,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-03 18:53:28,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-03 18:53:28,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-03 18:53:28,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-03 18:53:28,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-03 18:53:28,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-03 18:53:28,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-03 18:53:28,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-03 18:53:28,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-03 18:53:28,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-03 18:53:28,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-03 18:53:28,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-03 18:53:28,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-03 18:53:28,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-03 18:53:28,469 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0047588039714625595); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006733670025252104); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0047588039714625595); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-03 18:53:28,469 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-03 18:53:28,470 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=4.733604779029429E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0047588039714625595); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006733670025252104); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0047588039714625595); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-03 18:53:28,904 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 547 ms to try 5000 different iterations.  Found a solution that moves 230 regions; Going from a computed imbalance of 4.733604779029429E-4 to a new imbalance of 1.1924983476512634E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.158529131194229E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0019534423844256277); MoveCostFunction : (multiplier=7.0, imbalance=0.025555555555555557, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.158529131194229E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-03 18:53:28,905 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1264296761:83 , srv1570635312:83 , srv1716854338:83 , srv2120039846:83 , srv39253393:83 , srv67904614:83 , srv912414079:83 , srv1000439481:84 , srv1027430101:84 , srv1065202414:84 , srv119621826:84 , srv1204478931:84 , srv1267551397:84 , srv1341874019:84 , srv1367253232:84 , srv1413106800:84 , srv1462156661:84 , srv1472458599:84 , srv1517171282:84 , srv1604433647:84 , srv1620710337:84 , srv1622232158:84 , srv1633176335:84 , srv1675112244:84 , srv1687574320:84 , srv1717151055:84 , srv174742690:84 , srv1797720940:84 , srv1808794511:84 , srv1819772915:84 , srv1829098249:84 , srv1840734068:84 , srv1841590705:84 , srv2143753856:84 , srv2145053026:84 , srv278409005:84 , srv337208024:84 , srv371046896:84 , srv373842725:84 , srv403781194:84 , srv416221137:84 , srv469258617:84 , srv577670361:84 , srv626144805:84 , srv640409694:84 , srv693834011:84 , srv785437484:84 , srv802117795:84 , srv856589825:84 , srv889712869:84 , srv1028308513:85 , srv1079986837:85 , srv1118383304:85 , srv1123624951:85 , srv1130093103:85 , srv1148351270:85 , srv1154941282:85 , srv1164112296:85 , srv1200718078:85 , srv1212721765:85 , srv1319356539:85 , srv1320391385:85 , srv1346507235:85 , srv1438246720:85 , srv1458135232:85 , srv1585168174:85 , srv178331828:85 , srv1796113219:85 , srv1821188964:85 , srv1850626024:85 , srv1864917661:85 , srv1970140163:85 , srv2002687294:85 , srv2051399124:85 , srv2061052211:85 , srv2066923478:85 , srv2116427915:85 , srv218590600:85 , srv272197286:85 , srv332755048:85 , srv337783787:85 , srv378751723:85 , srv433786686:85 , srv46009038:85 , srv488156383:85 , srv502737949:85 , srv540037980:85 , srv564652720:85 , srv57806747:85 , srv605003136:85 , srv651690088:85 , srv693656483:85 , srv698277412:85 , srv710464330:85 , srv810510719:85 , srv887347300:85 , srv901473027:85 , srv905241:85 , srv942742178:85 , srv1581297441:642 }
2023-10-03 18:53:28,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-03 18:53:28,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv278409005=62, srv1716854338=37, srv651690088=84, srv1154941282=9, srv178331828=40, srv1581297441=29, srv905241=97, srv2143753856=58, srv1123624951=6, srv540037980=77, srv488156383=75, srv1829098249=46, srv1164112296=10, srv693656483=86, srv901473027=96, srv1319356539=17, srv1620710337=32, srv1864917661=50, srv785437484=90, srv1264296761=15, srv2145053026=59, srv337783787=65, srv46009038=73, srv1367253232=21, srv67904614=85, srv57806747=80, srv1458135232=24, srv1970140163=51, srv2051399124=53, srv2120039846=57, srv1148351270=8, srv119621826=11, srv640409694=83, srv698277412=88, srv174742690=39, srv889712869=95, srv1320391385=18, srv39253393=69, srv2066923478=55, srv1717151055=38, srv1413106800=22, srv1622232158=33, srv469258617=74, srv1346507235=20, srv337208024=64, srv416221137=71, srv856589825=93, srv626144805=82, srv1065202414=3, srv912414079=98, srv1267551397=16, srv1687574320=36, srv1633176335=34, srv1796113219=41, srv1079986837=4, srv1472458599=26, srv371046896=66, srv332755048=63, srv802117795=91, srv1819772915=44, srv887347300=94, srv1118383304=5, srv433786686=72, srv693834011=87, srv1341874019=19, srv378751723=68, srv1028308513=2, srv1808794511=43, srv1850626024=49, srv1821188964=45, srv1027430101=1, srv1570635312=28, srv1675112244=35, srv564652720=78, srv2116427915=56, srv218590600=60, srv1200718078=12, srv605003136=81, srv1204478931=13, srv1438246720=23, srv710464330=89, srv1462156661=25, srv2061052211=54, srv373842725=67, srv942742178=99, srv1517171282=27, srv1840734068=47, srv1000439481=0, srv1212721765=14, srv1841590705=48, srv1130093103=7, srv1585168174=30, srv1797720940=42, srv272197286=61, srv577670361=79, srv1604433647=31, srv2002687294=52, srv810510719=92, srv403781194=70, srv502737949=76} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-03 18:53:28,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-03 18:53:28,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-03 18:53:28,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-03 18:53:28,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-03 18:53:28,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-03 18:53:28,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-03 18:53:28,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-03 18:53:28,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-03 18:53:28,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-03 18:53:28,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-03 18:53:28,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-03 18:53:28,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-03 18:53:28,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-03 18:53:28,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-03 18:53:28,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-03 18:53:28,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-03 18:53:28,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-03 18:53:28,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-03 18:53:28,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-03 18:53:28,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-03 18:53:28,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-03 18:53:28,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-03 18:53:28,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-03 18:53:28,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-03 18:53:28,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-03 18:53:28,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-03 18:53:28,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-03 18:53:28,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-03 18:53:28,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-03 18:53:28,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-03 18:53:28,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-03 18:53:28,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-03 18:53:28,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-03 18:53:28,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-03 18:53:28,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-03 18:53:28,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-03 18:53:28,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-03 18:53:28,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-03 18:53:28,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-03 18:53:28,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-03 18:53:28,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-03 18:53:28,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-03 18:53:28,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-03 18:53:28,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-03 18:53:28,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-03 18:53:28,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-03 18:53:28,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-03 18:53:28,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-03 18:53:28,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-03 18:53:28,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-03 18:53:28,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-03 18:53:28,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-03 18:53:28,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-03 18:53:28,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-03 18:53:28,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-03 18:53:28,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-03 18:53:28,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-03 18:53:28,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-03 18:53:28,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-03 18:53:28,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-03 18:53:28,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-03 18:53:28,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-03 18:53:28,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-03 18:53:28,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-03 18:53:28,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-03 18:53:28,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-03 18:53:28,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-03 18:53:28,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-03 18:53:28,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-03 18:53:28,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-03 18:53:28,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-03 18:53:28,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-03 18:53:28,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-03 18:53:28,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-03 18:53:28,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-03 18:53:28,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-03 18:53:28,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-03 18:53:28,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-03 18:53:28,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-03 18:53:28,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-03 18:53:28,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-03 18:53:28,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-03 18:53:28,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-03 18:53:28,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-03 18:53:28,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-03 18:53:28,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-03 18:53:28,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-03 18:53:28,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-03 18:53:28,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-03 18:53:28,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-03 18:53:28,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-03 18:53:28,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-03 18:53:28,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-03 18:53:28,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-03 18:53:28,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-03 18:53:28,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-03 18:53:28,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-03 18:53:28,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-03 18:53:28,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-03 18:53:28,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-03 18:53:28,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-03 18:53:29,052 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.0314489054725146E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.158529131194229E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0019534423844256277); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.158529131194229E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-03 18:53:29,052 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1264296761:83 , srv1570635312:83 , srv1716854338:83 , srv2120039846:83 , srv39253393:83 , srv67904614:83 , srv912414079:83 , srv1000439481:84 , srv1027430101:84 , srv1065202414:84 , srv119621826:84 , srv1204478931:84 , srv1267551397:84 , srv1341874019:84 , srv1367253232:84 , srv1413106800:84 , srv1462156661:84 , srv1472458599:84 , srv1517171282:84 , srv1604433647:84 , srv1620710337:84 , srv1622232158:84 , srv1633176335:84 , srv1675112244:84 , srv1687574320:84 , srv1717151055:84 , srv174742690:84 , srv1797720940:84 , srv1808794511:84 , srv1819772915:84 , srv1829098249:84 , srv1840734068:84 , srv1841590705:84 , srv2143753856:84 , srv2145053026:84 , srv278409005:84 , srv337208024:84 , srv371046896:84 , srv373842725:84 , srv403781194:84 , srv416221137:84 , srv469258617:84 , srv577670361:84 , srv626144805:84 , srv640409694:84 , srv693834011:84 , srv785437484:84 , srv802117795:84 , srv856589825:84 , srv889712869:84 , srv1028308513:85 , srv1079986837:85 , srv1118383304:85 , srv1123624951:85 , srv1130093103:85 , srv1148351270:85 , srv1154941282:85 , srv1164112296:85 , srv1200718078:85 , srv1212721765:85 , srv1319356539:85 , srv1320391385:85 , srv1346507235:85 , srv1438246720:85 , srv1458135232:85 , srv1585168174:85 , srv178331828:85 , srv1796113219:85 , srv1821188964:85 , srv1850626024:85 , srv1864917661:85 , srv1970140163:85 , srv2002687294:85 , srv2051399124:85 , srv2061052211:85 , srv2066923478:85 , srv2116427915:85 , srv218590600:85 , srv272197286:85 , srv332755048:85 , srv337783787:85 , srv378751723:85 , srv433786686:85 , srv46009038:85 , srv488156383:85 , srv502737949:85 , srv540037980:85 , srv564652720:85 , srv57806747:85 , srv605003136:85 , srv651690088:85 , srv693656483:85 , srv698277412:85 , srv710464330:85 , srv810510719:85 , srv887347300:85 , srv901473027:85 , srv905241:85 , srv942742178:85 , srv1581297441:642 }
2023-10-03 18:53:29,087 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=297 (was 297), ProcessCount=170 (was 170), AvailableMemoryMB=8757 (was 8759)
2023-10-03 18:53:29,098 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=297, ProcessCount=171, AvailableMemoryMB=8756
2023-10-03 18:53:29,098 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-10-03 18:53:29,099 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-10-03 18:53:29,114 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1076237159:84 , srv1087593342:84 , srv1164885325:84 , srv1256052374:84 , srv128314048:84 , srv1351414107:84 , srv1470596831:84 , srv1472574999:84 , srv1516972305:84 , srv1615988360:84 , srv1757665047:84 , srv180048908:84 , srv183618504:84 , srv1964051224:84 , srv2033047508:84 , srv2138932866:84 , srv248758286:84 , srv269222237:84 , srv298424172:84 , srv309957360:84 , srv357246047:84 , srv419527361:84 , srv434255232:84 , srv450591835:84 , srv460493391:84 , srv770971121:84 , srv831326631:84 , srv905156775:84 , srv992893947:84 , srv638479265:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-10-03 18:53:29,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-03 18:53:29,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1472574999=7, srv1615988360=9, srv357246047=20, srv1076237159=0, srv992893947=29, srv269222237=17, srv1516972305=8, srv2138932866=15, srv460493391=24, srv770971121=26, srv2033047508=14, srv638479265=25, srv1964051224=13, srv298424172=18, srv248758286=16, srv434255232=22, srv1470596831=6, srv905156775=28, srv1757665047=10, srv309957360=19, srv1256052374=3, srv419527361=21, srv831326631=27, srv180048908=11, srv183618504=12, srv1164885325=2, srv128314048=4, srv1087593342=1, srv1351414107=5, srv450591835=23} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-03 18:53:29,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-03 18:53:29,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-03 18:53:29,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-03 18:53:29,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-03 18:53:29,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-03 18:53:29,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-03 18:53:29,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-03 18:53:29,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-03 18:53:29,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-03 18:53:29,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-03 18:53:29,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-03 18:53:29,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-03 18:53:29,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-03 18:53:29,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-03 18:53:29,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-03 18:53:29,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-03 18:53:29,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-03 18:53:29,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-03 18:53:29,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-03 18:53:29,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-03 18:53:29,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-03 18:53:29,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-03 18:53:29,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-03 18:53:29,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-03 18:53:29,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-03 18:53:29,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-03 18:53:29,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-03 18:53:29,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-03 18:53:29,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-03 18:53:29,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-03 18:53:29,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-10-03 18:53:29,160 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0820076906094476, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-03 18:53:29,161 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912451446819593, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0820076906094476, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-10-03 18:55:29,137 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6130872 different iterations.  Found a solution that moves 1850 regions; Going from a computed imbalance of 0.9912451446819593 to a new imbalance of 4.317992290299789E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6851851851851852, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-03 18:55:29,143 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1076237159:90 , srv1087593342:90 , srv1164885325:90 , srv1256052374:90 , srv128314048:90 , srv1351414107:90 , srv1470596831:90 , srv1472574999:90 , srv1516972305:90 , srv1615988360:90 , srv1757665047:90 , srv180048908:90 , srv183618504:90 , srv1964051224:90 , srv2033047508:90 , srv2138932866:90 , srv248758286:90 , srv269222237:90 , srv298424172:90 , srv309957360:90 , srv357246047:90 , srv419527361:90 , srv434255232:90 , srv450591835:90 , srv460493391:90 , srv638479265:90 , srv770971121:90 , srv831326631:90 , srv905156775:90 , srv992893947:90 }
2023-10-03 18:55:29,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-03 18:55:29,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1472574999=7, srv1615988360=9, srv357246047=20, srv1076237159=0, srv992893947=29, srv269222237=17, srv1516972305=8, srv2138932866=15, srv460493391=24, srv770971121=26, srv2033047508=14, srv638479265=25, srv1964051224=13, srv298424172=18, srv248758286=16, srv434255232=22, srv1470596831=6, srv905156775=28, srv1757665047=10, srv309957360=19, srv1256052374=3, srv419527361=21, srv831326631=27, srv180048908=11, srv183618504=12, srv1164885325=2, srv128314048=4, srv1087593342=1, srv1351414107=5, srv450591835=23} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-03 18:55:29,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-03 18:55:29,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-03 18:55:29,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-03 18:55:29,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-03 18:55:29,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-03 18:55:29,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-03 18:55:29,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-03 18:55:29,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-03 18:55:29,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-03 18:55:29,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-03 18:55:29,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-03 18:55:29,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-03 18:55:29,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-03 18:55:29,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-03 18:55:29,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-03 18:55:29,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-03 18:55:29,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-03 18:55:29,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-03 18:55:29,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-03 18:55:29,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-03 18:55:29,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-03 18:55:29,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-03 18:55:29,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-03 18:55:29,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-03 18:55:29,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-03 18:55:29,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-03 18:55:29,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-03 18:55:29,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-03 18:55:29,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-03 18:55:29,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-03 18:55:29,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-10-03 18:55:29,160 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-03 18:55:29,160 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1076237159:90 , srv1087593342:90 , srv1164885325:90 , srv1256052374:90 , srv128314048:90 , srv1351414107:90 , srv1470596831:90 , srv1472574999:90 , srv1516972305:90 , srv1615988360:90 , srv1757665047:90 , srv180048908:90 , srv183618504:90 , srv1964051224:90 , srv2033047508:90 , srv2138932866:90 , srv248758286:90 , srv269222237:90 , srv298424172:90 , srv309957360:90 , srv357246047:90 , srv419527361:90 , srv434255232:90 , srv450591835:90 , srv460493391:90 , srv638479265:90 , srv770971121:90 , srv831326631:90 , srv905156775:90 , srv992893947:90 }
2023-10-03 18:55:29,173 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=196 (was 297), ProcessCount=170 (was 171), AvailableMemoryMB=7460 (was 8756)