2023-10-23 15:53:28,552 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-10-23 15:53:28,860 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-10-23 15:53:28,971 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-10-23 15:53:29,042 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-10-23 15:53:29,053 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-10-23 15:53:29,068 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=391, ProcessCount=170, AvailableMemoryMB=8671
2023-10-23 15:53:29,071 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-10-23 15:53:29,071 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-10-23 15:53:29,195 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1002995850:84 , srv1032392954:84 , srv1046638504:84 , srv1066867111:84 , srv1109127669:84 , srv1118227286:84 , srv1141812139:84 , srv1143097009:84 , srv1151402897:84 , srv122009125:84 , srv1225247412:84 , srv1251601684:84 , srv1268385520:84 , srv128058872:84 , srv1304261737:84 , srv1312747909:84 , srv1313842568:84 , srv1326261518:84 , srv133271459:84 , srv1344419191:84 , srv1346647692:84 , srv1386643872:84 , srv1397151365:84 , srv1411507030:84 , srv1433213962:84 , srv1461984528:84 , srv1462929155:84 , srv1464900714:84 , srv1524179382:84 , srv1532725852:84 , srv1550032628:84 , srv1575385366:84 , srv1633819435:84 , srv1648794028:84 , srv1657146353:84 , srv1665724104:84 , srv1694902916:84 , srv1718919966:84 , srv1769517721:84 , srv178491203:84 , srv1785380897:84 , srv1787660065:84 , srv1809745683:84 , srv1872333896:84 , srv1927426374:84 , srv1942814583:84 , srv1946699815:84 , srv1962657362:84 , srv1999431421:84 , srv200232796:84 , srv2005581721:84 , srv2017124184:84 , srv2019045028:84 , srv2048304277:84 , srv2118191227:84 , srv2124844401:84 , srv2135244875:84 , srv2139067508:84 , srv2144313557:84 , srv2145295234:84 , srv214720915:84 , srv260613476:84 , srv266304012:84 , srv284159742:84 , srv291863668:84 , srv297886079:84 , srv297958703:84 , srv319516214:84 , srv32269825:84 , srv353320964:84 , srv422026455:84 , srv431523583:84 , srv484668784:84 , srv527826479:84 , srv548475466:84 , srv55575541:84 , srv586722766:84 , srv598702857:84 , srv614568367:84 , srv649722140:84 , srv650737149:84 , srv675074807:84 , srv697055969:84 , srv723833818:84 , srv738716827:84 , srv746474427:84 , srv799070919:84 , srv804792:84 , srv804801469:84 , srv828226272:84 , srv838498341:84 , srv85425072:84 , srv855993750:84 , srv886488831:84 , srv906491252:84 , srv933441090:84 , srv945595899:84 , srv951802787:84 , srv954565409:84 , srv230867202:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-10-23 15:53:29,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-23 15:53:29,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1464900714=27, srv1633819435=32, srv319516214=68, srv1927426374=44, srv2019045028=52, srv951802787=98, srv85425072=92, srv1344419191=19, srv2139067508=57, srv291863668=65, srv422026455=71, srv799070919=87, srv214720915=60, srv838498341=91, srv954565409=99, srv1326261518=17, srv1346647692=20, srv738716827=85, srv1872333896=43, srv1433213962=24, srv1397151365=22, srv527826479=74, srv1411507030=23, srv828226272=90, srv614568367=79, srv650737149=81, srv906491252=95, srv230867202=61, srv945595899=97, srv1032392954=1, srv2135244875=56, srv933441090=96, srv1109127669=4, srv2017124184=51, srv178491203=39, srv431523583=72, srv1386643872=21, srv2145295234=59, srv1532725852=29, srv1312747909=15, srv1066867111=3, srv1946699815=46, srv598702857=78, srv1268385520=12, srv1002995850=0, srv55575541=76, srv133271459=18, srv200232796=49, srv122009125=9, srv1787660065=41, srv1462929155=26, srv2118191227=54, srv1143097009=7, srv675074807=82, srv1718919966=37, srv586722766=77, srv1648794028=33, srv1999431421=48, srv649722140=80, srv128058872=13, srv1461984528=25, srv2124844401=55, srv1575385366=31, srv297958703=67, srv1942814583=45, srv484668784=73, srv1313842568=16, srv746474427=86, srv1657146353=34, srv886488831=94, srv1118227286=5, srv1962657362=47, srv1694902916=36, srv1769517721=38, srv1141812139=6, srv32269825=69, srv1665724104=35, srv2005581721=50, srv1524179382=28, srv1251601684=11, srv284159742=64, srv1809745683=42, srv2048304277=53, srv1304261737=14, srv353320964=70, srv697055969=83, srv855993750=93, srv297886079=66, srv1151402897=8, srv548475466=75, srv804792=88, srv266304012=63, srv1785380897=40, srv1225247412=10, srv260613476=62, srv723833818=84, srv1550032628=30, srv1046638504=2, srv804801469=89, srv2144313557=58} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-23 15:53:29,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-23 15:53:29,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-23 15:53:29,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-23 15:53:29,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-23 15:53:29,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-23 15:53:29,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-23 15:53:29,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-23 15:53:29,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-23 15:53:29,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-23 15:53:29,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-23 15:53:29,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-23 15:53:29,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-23 15:53:29,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-23 15:53:29,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-23 15:53:29,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-23 15:53:29,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-23 15:53:29,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-23 15:53:29,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-23 15:53:29,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-23 15:53:29,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-23 15:53:29,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-23 15:53:29,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-23 15:53:29,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-23 15:53:29,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-23 15:53:29,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-23 15:53:29,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-23 15:53:29,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-23 15:53:29,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-23 15:53:29,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-23 15:53:29,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-23 15:53:29,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-23 15:53:29,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-23 15:53:29,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-23 15:53:29,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-23 15:53:29,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-23 15:53:29,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-23 15:53:29,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-23 15:53:29,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-23 15:53:29,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-23 15:53:29,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-23 15:53:29,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-23 15:53:29,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-23 15:53:29,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-23 15:53:29,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-23 15:53:29,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-23 15:53:29,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-23 15:53:29,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-23 15:53:29,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-23 15:53:29,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-23 15:53:29,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-23 15:53:29,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-23 15:53:29,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-23 15:53:29,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-23 15:53:29,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-23 15:53:29,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-23 15:53:29,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-23 15:53:29,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-23 15:53:29,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-23 15:53:29,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-23 15:53:29,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-23 15:53:29,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-23 15:53:29,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-23 15:53:29,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-23 15:53:29,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-23 15:53:29,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-23 15:53:29,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-23 15:53:29,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-23 15:53:29,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-23 15:53:29,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-23 15:53:29,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-23 15:53:29,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-23 15:53:29,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-23 15:53:29,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-23 15:53:29,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-23 15:53:29,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-23 15:53:29,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-23 15:53:29,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-23 15:53:29,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-23 15:53:29,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-23 15:53:29,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-23 15:53:29,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-23 15:53:29,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-23 15:53:29,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-23 15:53:29,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-23 15:53:29,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-23 15:53:29,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-23 15:53:29,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-23 15:53:29,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-23 15:53:29,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-23 15:53:29,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-23 15:53:29,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-23 15:53:29,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-23 15:53:29,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-23 15:53:29,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-23 15:53:29,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-23 15:53:29,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-23 15:53:29,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-23 15:53:29,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-23 15:53:29,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-23 15:53:29,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-23 15:53:29,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-23 15:53:29,468 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-23 15:53:29,468 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-23 15:53:29,468 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-23 15:53:29,958 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 643 ms to try 5000 different iterations.  Found a solution that moves 4680 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1788847882421031. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03816628642700757, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.055186653992610496, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.52, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03816628642700757, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1745, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23683333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-23 15:53:29,980 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1633819435:81 , srv1151402897:82 , srv1946699815:82 , srv422026455:82 , srv1002995850:83 , srv1313842568:83 , srv1657146353:83 , srv1927426374:83 , srv2019045028:83 , srv2145295234:83 , srv297886079:83 , srv675074807:83 , srv697055969:83 , srv1326261518:84 , srv1411507030:84 , srv1769517721:84 , srv1785380897:84 , srv1872333896:84 , srv799070919:84 , srv804801469:84 , srv855993750:84 , srv1141812139:85 , srv1268385520:85 , srv128058872:85 , srv1304261737:85 , srv1312747909:85 , srv1461984528:85 , srv1462929155:85 , srv1575385366:85 , srv1648794028:85 , srv1694902916:85 , srv1962657362:85 , srv2139067508:85 , srv2144313557:85 , srv260613476:85 , srv319516214:85 , srv548475466:85 , srv723833818:85 , srv838498341:85 , srv1032392954:86 , srv1046638504:86 , srv1251601684:86 , srv1397151365:86 , srv178491203:86 , srv1787660065:86 , srv2017124184:86 , srv214720915:86 , srv266304012:86 , srv297958703:86 , srv353320964:86 , srv431523583:86 , srv55575541:86 , srv598702857:86 , srv906491252:86 , srv954565409:86 , srv1109127669:87 , srv1118227286:87 , srv1143097009:87 , srv1550032628:87 , srv1718919966:87 , srv1942814583:87 , srv1999431421:87 , srv2005581721:87 , srv2048304277:87 , srv2124844401:87 , srv133271459:88 , srv1386643872:88 , srv1532725852:88 , srv291863668:88 , srv484668784:88 , srv614568367:88 , srv738716827:88 , srv828226272:88 , srv85425072:88 , srv886488831:88 , srv951802787:88 , srv1225247412:89 , srv1524179382:89 , srv1665724104:89 , srv1809745683:89 , srv586722766:89 , srv804792:89 , srv945595899:89 , srv122009125:90 , srv1344419191:90 , srv1346647692:90 , srv200232796:90 , srv32269825:90 , srv746474427:90 , srv1433213962:91 , srv2118191227:91 , srv284159742:91 , srv933441090:91 , srv1066867111:92 , srv1464900714:92 , srv2135244875:92 , srv527826479:92 , srv649722140:92 , srv650737149:94 , srv230867202:429 }
2023-10-23 15:53:29,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-23 15:53:29,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1464900714=27, srv1633819435=32, srv319516214=68, srv1927426374=44, srv2019045028=52, srv951802787=98, srv85425072=92, srv1344419191=19, srv2139067508=57, srv291863668=65, srv422026455=71, srv799070919=87, srv214720915=60, srv838498341=91, srv954565409=99, srv1326261518=17, srv1346647692=20, srv738716827=85, srv1872333896=43, srv1433213962=24, srv1397151365=22, srv527826479=74, srv1411507030=23, srv828226272=90, srv614568367=79, srv650737149=81, srv906491252=95, srv230867202=61, srv945595899=97, srv1032392954=1, srv2135244875=56, srv933441090=96, srv1109127669=4, srv2017124184=51, srv178491203=39, srv431523583=72, srv1386643872=21, srv2145295234=59, srv1532725852=29, srv1312747909=15, srv1066867111=3, srv1946699815=46, srv598702857=78, srv1268385520=12, srv1002995850=0, srv55575541=76, srv133271459=18, srv200232796=49, srv122009125=9, srv1787660065=41, srv1462929155=26, srv2118191227=54, srv1143097009=7, srv675074807=82, srv1718919966=37, srv586722766=77, srv1648794028=33, srv1999431421=48, srv649722140=80, srv128058872=13, srv1461984528=25, srv2124844401=55, srv1575385366=31, srv297958703=67, srv1942814583=45, srv484668784=73, srv1313842568=16, srv746474427=86, srv1657146353=34, srv886488831=94, srv1118227286=5, srv1962657362=47, srv1694902916=36, srv1769517721=38, srv1141812139=6, srv32269825=69, srv1665724104=35, srv2005581721=50, srv1524179382=28, srv1251601684=11, srv284159742=64, srv1809745683=42, srv2048304277=53, srv1304261737=14, srv353320964=70, srv697055969=83, srv855993750=93, srv297886079=66, srv1151402897=8, srv548475466=75, srv804792=88, srv266304012=63, srv1785380897=40, srv1225247412=10, srv260613476=62, srv723833818=84, srv1550032628=30, srv1046638504=2, srv804801469=89, srv2144313557=58} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-23 15:53:30,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-23 15:53:30,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-23 15:53:30,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-23 15:53:30,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-23 15:53:30,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-23 15:53:30,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-23 15:53:30,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-23 15:53:30,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-23 15:53:30,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-23 15:53:30,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-23 15:53:30,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-23 15:53:30,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-23 15:53:30,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-23 15:53:30,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-23 15:53:30,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-23 15:53:30,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-23 15:53:30,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-23 15:53:30,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-23 15:53:30,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-23 15:53:30,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-23 15:53:30,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-23 15:53:30,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-23 15:53:30,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-23 15:53:30,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-23 15:53:30,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-23 15:53:30,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-23 15:53:30,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-23 15:53:30,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-23 15:53:30,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-23 15:53:30,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-23 15:53:30,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-23 15:53:30,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-23 15:53:30,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-23 15:53:30,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-23 15:53:30,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-23 15:53:30,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-23 15:53:30,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-23 15:53:30,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-23 15:53:30,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-23 15:53:30,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-23 15:53:30,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-23 15:53:30,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-23 15:53:30,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-23 15:53:30,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-23 15:53:30,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-23 15:53:30,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-23 15:53:30,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-23 15:53:30,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-23 15:53:30,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-23 15:53:30,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-23 15:53:30,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-23 15:53:30,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-23 15:53:30,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-23 15:53:30,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-23 15:53:30,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-23 15:53:30,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-23 15:53:30,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-23 15:53:30,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-23 15:53:30,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-23 15:53:30,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-23 15:53:30,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-23 15:53:30,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-23 15:53:30,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-23 15:53:30,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-23 15:53:30,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-23 15:53:30,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-23 15:53:30,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-23 15:53:30,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-23 15:53:30,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-23 15:53:30,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-23 15:53:30,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-23 15:53:30,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-23 15:53:30,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-23 15:53:30,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-23 15:53:30,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-23 15:53:30,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-23 15:53:30,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-23 15:53:30,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-23 15:53:30,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-23 15:53:30,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-23 15:53:30,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-23 15:53:30,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-23 15:53:30,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-23 15:53:30,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-23 15:53:30,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-23 15:53:30,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-23 15:53:30,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-23 15:53:30,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-23 15:53:30,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-23 15:53:30,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-23 15:53:30,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-23 15:53:30,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-23 15:53:30,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-23 15:53:30,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-23 15:53:30,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-23 15:53:30,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-23 15:53:30,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-23 15:53:30,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-23 15:53:30,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-23 15:53:30,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-23 15:53:30,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-23 15:53:30,170 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03816628642700757, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.055186653992610496, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03816628642700757, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1745, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23683333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-23 15:53:30,170 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-23 15:53:30,170 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17885201818169455, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03816628642700757, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.055186653992610496, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03816628642700757, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1745, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23683333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-23 15:53:30,483 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 447 ms to try 5000 different iterations.  Found a solution that moves 2576 regions; Going from a computed imbalance of 0.17885201818169455 to a new imbalance of 0.014416239369947941. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020259570104534576); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.030273723201785348, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2862222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020259570104534576); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01125); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.044833333333333336, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-23 15:53:30,493 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv650737149:76 , srv1066867111:80 , srv122009125:80 , srv1433213962:80 , srv1464900714:80 , srv2118191227:81 , srv2135244875:81 , srv527826479:81 , srv649722140:81 , srv746474427:81 , srv1386643872:82 , srv933441090:82 , srv133271459:83 , srv1665724104:83 , srv200232796:83 , srv32269825:83 , srv738716827:83 , srv85425072:83 , srv945595899:83 , srv951802787:83 , srv1344419191:84 , srv1524179382:84 , srv1999431421:84 , srv2048304277:84 , srv284159742:84 , srv291863668:84 , srv431523583:84 , srv586722766:84 , srv804792:84 , srv886488831:84 , srv1032392954:85 , srv1109127669:85 , srv1225247412:85 , srv1346647692:85 , srv1461984528:85 , srv1532725852:85 , srv1718919966:85 , srv1809745683:85 , srv1962657362:85 , srv598702857:85 , srv723833818:85 , srv828226272:85 , srv1046638504:86 , srv1118227286:86 , srv1143097009:86 , srv1251601684:86 , srv1268385520:86 , srv128058872:86 , srv1648794028:86 , srv1787660065:86 , srv2005581721:86 , srv2017124184:86 , srv2139067508:86 , srv260613476:86 , srv297958703:86 , srv319516214:86 , srv353320964:86 , srv484668784:86 , srv548475466:86 , srv55575541:86 , srv799070919:86 , srv906491252:86 , srv1141812139:87 , srv1304261737:87 , srv1312747909:87 , srv1397151365:87 , srv1462929155:87 , srv1550032628:87 , srv178491203:87 , srv1927426374:87 , srv1942814583:87 , srv214720915:87 , srv838498341:87 , srv954565409:87 , srv1326261518:88 , srv1575385366:88 , srv1657146353:88 , srv1694902916:88 , srv1946699815:88 , srv2124844401:88 , srv2144313557:88 , srv2145295234:88 , srv422026455:88 , srv614568367:88 , srv804801469:88 , srv1151402897:89 , srv1313842568:89 , srv1633819435:89 , srv1769517721:89 , srv1785380897:89 , srv1872333896:89 , srv697055969:89 , srv855993750:89 , srv1002995850:90 , srv1411507030:90 , srv266304012:90 , srv675074807:90 , srv2019045028:91 , srv297886079:91 , srv230867202:525 }
2023-10-23 15:53:30,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-23 15:53:30,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1464900714=27, srv1633819435=32, srv319516214=68, srv1927426374=44, srv2019045028=52, srv951802787=98, srv85425072=92, srv1344419191=19, srv2139067508=57, srv291863668=65, srv422026455=71, srv799070919=87, srv214720915=60, srv838498341=91, srv954565409=99, srv1326261518=17, srv1346647692=20, srv738716827=85, srv1872333896=43, srv1433213962=24, srv1397151365=22, srv527826479=74, srv1411507030=23, srv828226272=90, srv614568367=79, srv650737149=81, srv906491252=95, srv230867202=61, srv945595899=97, srv1032392954=1, srv2135244875=56, srv933441090=96, srv1109127669=4, srv2017124184=51, srv178491203=39, srv431523583=72, srv1386643872=21, srv2145295234=59, srv1532725852=29, srv1312747909=15, srv1066867111=3, srv1946699815=46, srv598702857=78, srv1268385520=12, srv1002995850=0, srv55575541=76, srv133271459=18, srv200232796=49, srv122009125=9, srv1787660065=41, srv1462929155=26, srv2118191227=54, srv1143097009=7, srv675074807=82, srv1718919966=37, srv586722766=77, srv1648794028=33, srv1999431421=48, srv649722140=80, srv128058872=13, srv1461984528=25, srv2124844401=55, srv1575385366=31, srv297958703=67, srv1942814583=45, srv484668784=73, srv1313842568=16, srv746474427=86, srv1657146353=34, srv886488831=94, srv1118227286=5, srv1962657362=47, srv1694902916=36, srv1769517721=38, srv1141812139=6, srv32269825=69, srv1665724104=35, srv2005581721=50, srv1524179382=28, srv1251601684=11, srv284159742=64, srv1809745683=42, srv2048304277=53, srv1304261737=14, srv353320964=70, srv697055969=83, srv855993750=93, srv297886079=66, srv1151402897=8, srv548475466=75, srv804792=88, srv266304012=63, srv1785380897=40, srv1225247412=10, srv260613476=62, srv723833818=84, srv1550032628=30, srv1046638504=2, srv804801469=89, srv2144313557=58} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-23 15:53:30,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-23 15:53:30,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-23 15:53:30,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-23 15:53:30,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-23 15:53:30,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-23 15:53:30,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-23 15:53:30,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-23 15:53:30,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-23 15:53:30,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-23 15:53:30,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-23 15:53:30,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-23 15:53:30,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-23 15:53:30,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-23 15:53:30,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-23 15:53:30,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-23 15:53:30,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-23 15:53:30,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-23 15:53:30,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-23 15:53:30,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-23 15:53:30,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-23 15:53:30,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-23 15:53:30,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-23 15:53:30,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-23 15:53:30,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-23 15:53:30,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-23 15:53:30,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-23 15:53:30,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-23 15:53:30,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-23 15:53:30,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-23 15:53:30,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-23 15:53:30,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-23 15:53:30,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-23 15:53:30,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-23 15:53:30,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-23 15:53:30,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-23 15:53:30,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-23 15:53:30,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-23 15:53:30,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-23 15:53:30,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-23 15:53:30,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-23 15:53:30,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-23 15:53:30,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-23 15:53:30,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-23 15:53:30,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-23 15:53:30,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-23 15:53:30,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-23 15:53:30,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-23 15:53:30,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-23 15:53:30,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-23 15:53:30,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-23 15:53:30,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-23 15:53:30,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-23 15:53:30,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-23 15:53:30,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-23 15:53:30,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-23 15:53:30,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-23 15:53:30,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-23 15:53:30,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-23 15:53:30,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-23 15:53:30,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-23 15:53:30,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-23 15:53:30,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-23 15:53:30,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-23 15:53:30,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-23 15:53:30,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-23 15:53:30,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-23 15:53:30,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-23 15:53:30,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-23 15:53:30,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-23 15:53:30,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-23 15:53:30,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-23 15:53:30,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-23 15:53:30,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-23 15:53:30,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-23 15:53:30,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-23 15:53:30,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-23 15:53:30,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-23 15:53:30,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-23 15:53:30,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-23 15:53:30,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-23 15:53:30,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-23 15:53:30,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-23 15:53:30,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-23 15:53:30,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-23 15:53:30,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-23 15:53:30,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-23 15:53:30,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-23 15:53:30,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-23 15:53:30,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-23 15:53:30,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-23 15:53:30,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-23 15:53:30,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-23 15:53:30,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-23 15:53:30,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-23 15:53:30,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-23 15:53:30,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-23 15:53:30,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-23 15:53:30,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-23 15:53:30,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-23 15:53:30,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-23 15:53:30,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-23 15:53:30,704 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020259570104534576); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.030273723201785348, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020259570104534576); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01125); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.044833333333333336, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-23 15:53:30,705 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-23 15:53:30,705 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.014398201832423923, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020259570104534576); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.030273723201785348, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020259570104534576); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01125); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.044833333333333336, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-23 15:53:31,027 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 466 ms to try 5000 different iterations.  Found a solution that moves 970 regions; Going from a computed imbalance of 0.014398201832423923 to a new imbalance of 5.056272879789024E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0038972973071139195); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00831479419283098); MoveCostFunction : (multiplier=7.0, imbalance=0.10777777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0038972973071139195); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004083333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-23 15:53:31,030 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv266304012:81 , srv2124844401:82 , srv1143097009:83 , srv1344419191:83 , srv1346647692:83 , srv284159742:83 , srv297886079:83 , srv614568367:83 , srv1141812139:84 , srv1225247412:84 , srv1397151365:84 , srv1550032628:84 , srv1575385366:84 , srv1694902916:84 , srv1809745683:84 , srv1872333896:84 , srv1942814583:84 , srv2019045028:84 , srv32269825:84 , srv484668784:84 , srv586722766:84 , srv675074807:84 , srv804792:84 , srv828226272:84 , srv855993750:84 , srv933441090:84 , srv954565409:84 , srv1002995850:85 , srv1046638504:85 , srv1066867111:85 , srv1109127669:85 , srv1118227286:85 , srv1251601684:85 , srv1313842568:85 , srv1326261518:85 , srv1411507030:85 , srv1462929155:85 , srv1524179382:85 , srv1532725852:85 , srv1769517721:85 , srv178491203:85 , srv1785380897:85 , srv200232796:85 , srv2005581721:85 , srv2135244875:85 , srv2144313557:85 , srv214720915:85 , srv527826479:85 , srv649722140:85 , srv804801469:85 , srv838498341:85 , srv906491252:85 , srv1032392954:86 , srv1268385520:86 , srv128058872:86 , srv1312747909:86 , srv1464900714:86 , srv1648794028:86 , srv1657146353:86 , srv1665724104:86 , srv1718919966:86 , srv1999431421:86 , srv2145295234:86 , srv291863668:86 , srv297958703:86 , srv353320964:86 , srv55575541:86 , srv598702857:86 , srv697055969:86 , srv738716827:86 , srv886488831:86 , srv945595899:86 , srv1151402897:87 , srv122009125:87 , srv1304261737:87 , srv133271459:87 , srv1386643872:87 , srv1433213962:87 , srv1633819435:87 , srv1787660065:87 , srv1946699815:87 , srv2017124184:87 , srv2048304277:87 , srv2118191227:87 , srv2139067508:87 , srv319516214:87 , srv548475466:87 , srv723833818:87 , srv746474427:87 , srv799070919:87 , srv85425072:87 , srv951802787:87 , srv1962657362:88 , srv260613476:88 , srv422026455:88 , srv650737149:88 , srv1461984528:89 , srv1927426374:89 , srv431523583:90 , srv230867202:538 }
2023-10-23 15:53:31,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-23 15:53:31,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1464900714=27, srv1633819435=32, srv319516214=68, srv1927426374=44, srv2019045028=52, srv951802787=98, srv85425072=92, srv1344419191=19, srv2139067508=57, srv291863668=65, srv422026455=71, srv799070919=87, srv214720915=60, srv838498341=91, srv954565409=99, srv1326261518=17, srv1346647692=20, srv738716827=85, srv1872333896=43, srv1433213962=24, srv1397151365=22, srv527826479=74, srv1411507030=23, srv828226272=90, srv614568367=79, srv650737149=81, srv906491252=95, srv230867202=61, srv945595899=97, srv1032392954=1, srv2135244875=56, srv933441090=96, srv1109127669=4, srv2017124184=51, srv178491203=39, srv431523583=72, srv1386643872=21, srv2145295234=59, srv1532725852=29, srv1312747909=15, srv1066867111=3, srv1946699815=46, srv598702857=78, srv1268385520=12, srv1002995850=0, srv55575541=76, srv133271459=18, srv200232796=49, srv122009125=9, srv1787660065=41, srv1462929155=26, srv2118191227=54, srv1143097009=7, srv675074807=82, srv1718919966=37, srv586722766=77, srv1648794028=33, srv1999431421=48, srv649722140=80, srv128058872=13, srv1461984528=25, srv2124844401=55, srv1575385366=31, srv297958703=67, srv1942814583=45, srv484668784=73, srv1313842568=16, srv746474427=86, srv1657146353=34, srv886488831=94, srv1118227286=5, srv1962657362=47, srv1694902916=36, srv1769517721=38, srv1141812139=6, srv32269825=69, srv1665724104=35, srv2005581721=50, srv1524179382=28, srv1251601684=11, srv284159742=64, srv1809745683=42, srv2048304277=53, srv1304261737=14, srv353320964=70, srv697055969=83, srv855993750=93, srv297886079=66, srv1151402897=8, srv548475466=75, srv804792=88, srv266304012=63, srv1785380897=40, srv1225247412=10, srv260613476=62, srv723833818=84, srv1550032628=30, srv1046638504=2, srv804801469=89, srv2144313557=58} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-23 15:53:31,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-23 15:53:31,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-23 15:53:31,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-23 15:53:31,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-23 15:53:31,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-23 15:53:31,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-23 15:53:31,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-23 15:53:31,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-23 15:53:31,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-23 15:53:31,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-23 15:53:31,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-23 15:53:31,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-23 15:53:31,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-23 15:53:31,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-23 15:53:31,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-23 15:53:31,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-23 15:53:31,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-23 15:53:31,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-23 15:53:31,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-23 15:53:31,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-23 15:53:31,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-23 15:53:31,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-23 15:53:31,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-23 15:53:31,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-23 15:53:31,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-23 15:53:31,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-23 15:53:31,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-23 15:53:31,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-23 15:53:31,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-23 15:53:31,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-23 15:53:31,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-23 15:53:31,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-23 15:53:31,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-23 15:53:31,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-23 15:53:31,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-23 15:53:31,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-23 15:53:31,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-23 15:53:31,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-23 15:53:31,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-23 15:53:31,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-23 15:53:31,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-23 15:53:31,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-23 15:53:31,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-23 15:53:31,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-23 15:53:31,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-23 15:53:31,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-23 15:53:31,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-23 15:53:31,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-23 15:53:31,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-23 15:53:31,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-23 15:53:31,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-23 15:53:31,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-23 15:53:31,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-23 15:53:31,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-23 15:53:31,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-23 15:53:31,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-23 15:53:31,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-23 15:53:31,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-23 15:53:31,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-23 15:53:31,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-23 15:53:31,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-23 15:53:31,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-23 15:53:31,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-23 15:53:31,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-23 15:53:31,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-23 15:53:31,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-23 15:53:31,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-23 15:53:31,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-23 15:53:31,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-23 15:53:31,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-23 15:53:31,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-23 15:53:31,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-23 15:53:31,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-23 15:53:31,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-23 15:53:31,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-23 15:53:31,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-23 15:53:31,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-23 15:53:31,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-23 15:53:31,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-23 15:53:31,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-23 15:53:31,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-23 15:53:31,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-23 15:53:31,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-23 15:53:31,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-23 15:53:31,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-23 15:53:31,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-23 15:53:31,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-23 15:53:31,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-23 15:53:31,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-23 15:53:31,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-23 15:53:31,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-23 15:53:31,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-23 15:53:31,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-23 15:53:31,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-23 15:53:31,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-23 15:53:31,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-23 15:53:31,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-23 15:53:31,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-23 15:53:31,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-23 15:53:31,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-23 15:53:31,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-23 15:53:31,232 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0038972973071139195); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00831479419283098); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0038972973071139195); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004083333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-23 15:53:31,232 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-23 15:53:31,233 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=4.988352028087551E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0038972973071139195); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00831479419283098); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0038972973071139195); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004083333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-23 15:53:31,612 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 526 ms to try 5000 different iterations.  Found a solution that moves 261 regions; Going from a computed imbalance of 4.988352028087551E-4 to a new imbalance of 2.0777363348048414E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0020651563939657444); MoveCostFunction : (multiplier=7.0, imbalance=0.029, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-23 15:53:31,613 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1461984528:82 , srv431523583:82 , srv133271459:83 , srv1346647692:83 , srv1787660065:83 , srv1927426374:83 , srv2017124184:83 , srv2118191227:83 , srv260613476:83 , srv614568367:83 , srv650737149:83 , srv1002995850:84 , srv1032392954:84 , srv1118227286:84 , srv122009125:84 , srv1225247412:84 , srv1304261737:84 , srv1312747909:84 , srv1326261518:84 , srv1411507030:84 , srv1433213962:84 , srv1464900714:84 , srv1524179382:84 , srv1532725852:84 , srv1550032628:84 , srv1665724104:84 , srv1718919966:84 , srv1769517721:84 , srv178491203:84 , srv1785380897:84 , srv1809745683:84 , srv1942814583:84 , srv1962657362:84 , srv200232796:84 , srv2005581721:84 , srv2019045028:84 , srv2048304277:84 , srv2135244875:84 , srv2139067508:84 , srv2144313557:84 , srv284159742:84 , srv291863668:84 , srv297958703:84 , srv319516214:84 , srv353320964:84 , srv422026455:84 , srv484668784:84 , srv527826479:84 , srv548475466:84 , srv55575541:84 , srv697055969:84 , srv746474427:84 , srv85425072:84 , srv886488831:84 , srv945595899:84 , srv951802787:84 , srv1046638504:85 , srv1066867111:85 , srv1109127669:85 , srv1151402897:85 , srv1251601684:85 , srv1268385520:85 , srv128058872:85 , srv1313842568:85 , srv1344419191:85 , srv1386643872:85 , srv1397151365:85 , srv1462929155:85 , srv1633819435:85 , srv1648794028:85 , srv1657146353:85 , srv1694902916:85 , srv1946699815:85 , srv1999431421:85 , srv2124844401:85 , srv2145295234:85 , srv214720915:85 , srv266304012:85 , srv297886079:85 , srv32269825:85 , srv586722766:85 , srv598702857:85 , srv649722140:85 , srv675074807:85 , srv723833818:85 , srv738716827:85 , srv799070919:85 , srv804792:85 , srv804801469:85 , srv828226272:85 , srv838498341:85 , srv855993750:85 , srv906491252:85 , srv933441090:85 , srv1141812139:86 , srv1143097009:86 , srv1575385366:86 , srv1872333896:86 , srv954565409:86 , srv230867202:649 }
2023-10-23 15:53:31,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-23 15:53:31,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1464900714=27, srv1633819435=32, srv319516214=68, srv1927426374=44, srv2019045028=52, srv951802787=98, srv85425072=92, srv1344419191=19, srv2139067508=57, srv291863668=65, srv422026455=71, srv799070919=87, srv214720915=60, srv838498341=91, srv954565409=99, srv1326261518=17, srv1346647692=20, srv738716827=85, srv1872333896=43, srv1433213962=24, srv1397151365=22, srv527826479=74, srv1411507030=23, srv828226272=90, srv614568367=79, srv650737149=81, srv906491252=95, srv230867202=61, srv945595899=97, srv1032392954=1, srv2135244875=56, srv933441090=96, srv1109127669=4, srv2017124184=51, srv178491203=39, srv431523583=72, srv1386643872=21, srv2145295234=59, srv1532725852=29, srv1312747909=15, srv1066867111=3, srv1946699815=46, srv598702857=78, srv1268385520=12, srv1002995850=0, srv55575541=76, srv133271459=18, srv200232796=49, srv122009125=9, srv1787660065=41, srv1462929155=26, srv2118191227=54, srv1143097009=7, srv675074807=82, srv1718919966=37, srv586722766=77, srv1648794028=33, srv1999431421=48, srv649722140=80, srv128058872=13, srv1461984528=25, srv2124844401=55, srv1575385366=31, srv297958703=67, srv1942814583=45, srv484668784=73, srv1313842568=16, srv746474427=86, srv1657146353=34, srv886488831=94, srv1118227286=5, srv1962657362=47, srv1694902916=36, srv1769517721=38, srv1141812139=6, srv32269825=69, srv1665724104=35, srv2005581721=50, srv1524179382=28, srv1251601684=11, srv284159742=64, srv1809745683=42, srv2048304277=53, srv1304261737=14, srv353320964=70, srv697055969=83, srv855993750=93, srv297886079=66, srv1151402897=8, srv548475466=75, srv804792=88, srv266304012=63, srv1785380897=40, srv1225247412=10, srv260613476=62, srv723833818=84, srv1550032628=30, srv1046638504=2, srv804801469=89, srv2144313557=58} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-23 15:53:31,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-23 15:53:31,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-23 15:53:31,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-23 15:53:31,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-23 15:53:31,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-23 15:53:31,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-23 15:53:31,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-23 15:53:31,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-23 15:53:31,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-23 15:53:31,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-23 15:53:31,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-23 15:53:31,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-23 15:53:31,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-23 15:53:31,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-23 15:53:31,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-23 15:53:31,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-23 15:53:31,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-23 15:53:31,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-23 15:53:31,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-23 15:53:31,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-23 15:53:31,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-23 15:53:31,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-23 15:53:31,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-23 15:53:31,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-23 15:53:31,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-23 15:53:31,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-23 15:53:31,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-23 15:53:31,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-23 15:53:31,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-23 15:53:31,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-23 15:53:31,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-23 15:53:31,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-23 15:53:31,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-23 15:53:31,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-23 15:53:31,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-23 15:53:31,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-23 15:53:31,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-23 15:53:31,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-23 15:53:31,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-23 15:53:31,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-23 15:53:31,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-23 15:53:31,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-23 15:53:31,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-23 15:53:31,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-23 15:53:31,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-23 15:53:31,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-23 15:53:31,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-23 15:53:31,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-23 15:53:31,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-23 15:53:31,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-23 15:53:31,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-23 15:53:31,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-23 15:53:31,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-23 15:53:31,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-23 15:53:31,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-23 15:53:31,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-23 15:53:31,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-23 15:53:31,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-23 15:53:31,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-23 15:53:31,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-23 15:53:31,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-23 15:53:31,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-23 15:53:31,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-23 15:53:31,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-23 15:53:31,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-23 15:53:31,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-23 15:53:31,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-23 15:53:31,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-23 15:53:31,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-23 15:53:31,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-23 15:53:31,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-23 15:53:31,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-23 15:53:31,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-23 15:53:31,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-23 15:53:31,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-23 15:53:31,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-23 15:53:31,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-23 15:53:31,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-23 15:53:31,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-23 15:53:31,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-23 15:53:31,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-23 15:53:31,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-23 15:53:31,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-23 15:53:31,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-23 15:53:31,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-23 15:53:31,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-23 15:53:31,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-23 15:53:31,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-23 15:53:31,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-23 15:53:31,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-23 15:53:31,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-23 15:53:31,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-23 15:53:31,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-23 15:53:31,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-23 15:53:31,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-23 15:53:31,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-23 15:53:31,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-23 15:53:31,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-23 15:53:31,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-23 15:53:31,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-23 15:53:31,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-23 15:53:31,742 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.894980228680261E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0020651563939657444); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-23 15:53:31,742 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1461984528:82 , srv431523583:82 , srv133271459:83 , srv1346647692:83 , srv1787660065:83 , srv1927426374:83 , srv2017124184:83 , srv2118191227:83 , srv260613476:83 , srv614568367:83 , srv650737149:83 , srv1002995850:84 , srv1032392954:84 , srv1118227286:84 , srv122009125:84 , srv1225247412:84 , srv1304261737:84 , srv1312747909:84 , srv1326261518:84 , srv1411507030:84 , srv1433213962:84 , srv1464900714:84 , srv1524179382:84 , srv1532725852:84 , srv1550032628:84 , srv1665724104:84 , srv1718919966:84 , srv1769517721:84 , srv178491203:84 , srv1785380897:84 , srv1809745683:84 , srv1942814583:84 , srv1962657362:84 , srv200232796:84 , srv2005581721:84 , srv2019045028:84 , srv2048304277:84 , srv2135244875:84 , srv2139067508:84 , srv2144313557:84 , srv284159742:84 , srv291863668:84 , srv297958703:84 , srv319516214:84 , srv353320964:84 , srv422026455:84 , srv484668784:84 , srv527826479:84 , srv548475466:84 , srv55575541:84 , srv697055969:84 , srv746474427:84 , srv85425072:84 , srv886488831:84 , srv945595899:84 , srv951802787:84 , srv1046638504:85 , srv1066867111:85 , srv1109127669:85 , srv1151402897:85 , srv1251601684:85 , srv1268385520:85 , srv128058872:85 , srv1313842568:85 , srv1344419191:85 , srv1386643872:85 , srv1397151365:85 , srv1462929155:85 , srv1633819435:85 , srv1648794028:85 , srv1657146353:85 , srv1694902916:85 , srv1946699815:85 , srv1999431421:85 , srv2124844401:85 , srv2145295234:85 , srv214720915:85 , srv266304012:85 , srv297886079:85 , srv32269825:85 , srv586722766:85 , srv598702857:85 , srv649722140:85 , srv675074807:85 , srv723833818:85 , srv738716827:85 , srv799070919:85 , srv804792:85 , srv804801469:85 , srv828226272:85 , srv838498341:85 , srv855993750:85 , srv906491252:85 , srv933441090:85 , srv1141812139:86 , srv1143097009:86 , srv1575385366:86 , srv1872333896:86 , srv954565409:86 , srv230867202:649 }
2023-10-23 15:53:31,770 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=391 (was 391), ProcessCount=170 (was 170), AvailableMemoryMB=8162 (was 8671)
2023-10-23 15:53:31,780 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=391, ProcessCount=170, AvailableMemoryMB=8162
2023-10-23 15:53:31,781 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-10-23 15:53:31,781 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-10-23 15:53:31,789 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1069397522:84 , srv1228861580:84 , srv1275771236:84 , srv1352827651:84 , srv1357132861:84 , srv142280179:84 , srv1505945246:84 , srv1523020618:84 , srv1531766252:84 , srv1804035351:84 , srv1963610882:84 , srv196395036:84 , srv201123619:84 , srv2047446520:84 , srv2100242979:84 , srv2113184930:84 , srv448584254:84 , srv45716610:84 , srv572921933:84 , srv584385437:84 , srv638433438:84 , srv664575962:84 , srv702051699:84 , srv70844897:84 , srv727129657:84 , srv728634107:84 , srv740714928:84 , srv755061553:84 , srv952370005:84 , srv998163478:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-10-23 15:53:31,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-23 15:53:31,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2047446520=13, srv740714928=26, srv998163478=29, srv638433438=20, srv1531766252=8, srv572921933=18, srv1275771236=2, srv142280179=5, srv1069397522=0, srv201123619=12, srv196395036=11, srv727129657=24, srv2100242979=14, srv664575962=21, srv952370005=28, srv1963610882=10, srv702051699=22, srv1505945246=6, srv70844897=23, srv584385437=19, srv1804035351=9, srv728634107=25, srv2113184930=15, srv1357132861=4, srv45716610=17, srv1352827651=3, srv755061553=27, srv1228861580=1, srv1523020618=7, srv448584254=16} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-23 15:53:31,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-23 15:53:31,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-23 15:53:31,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-23 15:53:31,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-23 15:53:31,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-23 15:53:31,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-23 15:53:31,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-23 15:53:31,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-23 15:53:31,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-23 15:53:31,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-23 15:53:31,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-23 15:53:31,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-23 15:53:31,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-23 15:53:31,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-23 15:53:31,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-23 15:53:31,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-23 15:53:31,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-23 15:53:31,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-23 15:53:31,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-23 15:53:31,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-23 15:53:31,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-23 15:53:31,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-23 15:53:31,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-23 15:53:31,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-23 15:53:31,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-23 15:53:31,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-23 15:53:31,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-23 15:53:31,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-23 15:53:31,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-23 15:53:31,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-23 15:53:31,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-10-23 15:53:31,814 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1117884079814697, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-23 15:53:31,814 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912545284887602, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1117884079814697, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-10-23 15:55:31,803 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5597336 different iterations.  Found a solution that moves 1850 regions; Going from a computed imbalance of 0.9912545284887602 to a new imbalance of 4.317992290299789E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6851851851851852, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-23 15:55:31,810 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1069397522:90 , srv1228861580:90 , srv1275771236:90 , srv1352827651:90 , srv1357132861:90 , srv142280179:90 , srv1505945246:90 , srv1523020618:90 , srv1531766252:90 , srv1804035351:90 , srv1963610882:90 , srv196395036:90 , srv201123619:90 , srv2047446520:90 , srv2100242979:90 , srv2113184930:90 , srv448584254:90 , srv45716610:90 , srv572921933:90 , srv584385437:90 , srv638433438:90 , srv664575962:90 , srv702051699:90 , srv70844897:90 , srv727129657:90 , srv728634107:90 , srv740714928:90 , srv755061553:90 , srv952370005:90 , srv998163478:90 }
2023-10-23 15:55:31,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-23 15:55:31,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2047446520=13, srv740714928=26, srv998163478=29, srv638433438=20, srv1531766252=8, srv572921933=18, srv1275771236=2, srv142280179=5, srv1069397522=0, srv201123619=12, srv196395036=11, srv727129657=24, srv2100242979=14, srv664575962=21, srv952370005=28, srv1963610882=10, srv702051699=22, srv1505945246=6, srv70844897=23, srv584385437=19, srv1804035351=9, srv728634107=25, srv2113184930=15, srv1357132861=4, srv45716610=17, srv1352827651=3, srv755061553=27, srv1228861580=1, srv1523020618=7, srv448584254=16} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-23 15:55:31,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-23 15:55:31,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-23 15:55:31,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-23 15:55:31,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-23 15:55:31,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-23 15:55:31,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-23 15:55:31,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-23 15:55:31,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-23 15:55:31,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-23 15:55:31,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-23 15:55:31,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-23 15:55:31,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-23 15:55:31,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-23 15:55:31,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-23 15:55:31,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-23 15:55:31,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-23 15:55:31,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-23 15:55:31,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-23 15:55:31,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-23 15:55:31,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-23 15:55:31,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-23 15:55:31,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-23 15:55:31,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-23 15:55:31,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-23 15:55:31,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-23 15:55:31,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-23 15:55:31,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-23 15:55:31,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-23 15:55:31,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-23 15:55:31,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-23 15:55:31,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-10-23 15:55:31,828 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-23 15:55:31,828 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1069397522:90 , srv1228861580:90 , srv1275771236:90 , srv1352827651:90 , srv1357132861:90 , srv142280179:90 , srv1505945246:90 , srv1523020618:90 , srv1531766252:90 , srv1804035351:90 , srv1963610882:90 , srv196395036:90 , srv201123619:90 , srv2047446520:90 , srv2100242979:90 , srv2113184930:90 , srv448584254:90 , srv45716610:90 , srv572921933:90 , srv584385437:90 , srv638433438:90 , srv664575962:90 , srv702051699:90 , srv70844897:90 , srv727129657:90 , srv728634107:90 , srv740714928:90 , srv755061553:90 , srv952370005:90 , srv998163478:90 }
2023-10-23 15:55:31,842 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=268 (was 391), ProcessCount=166 (was 170), AvailableMemoryMB=8069 (was 8162)