2023-05-12 20:53:24,289 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-05-12 20:53:24,620 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-05-12 20:53:24,742 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-05-12 20:53:24,802 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-12 20:53:24,810 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-05-12 20:53:24,827 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=345, ProcessCount=171, AvailableMemoryMB=10354
2023-05-12 20:53:24,830 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-12 20:53:24,830 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-05-12 20:53:24,976 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1007949230:84 , srv1008975073:84 , srv1038764834:84 , srv1055231135:84 , srv1122827778:84 , srv1130896070:84 , srv1152405931:84 , srv1172937503:84 , srv1199866789:84 , srv1215282646:84 , srv1255890433:84 , srv1259112757:84 , srv1270681259:84 , srv1271521098:84 , srv1284709569:84 , srv1306662132:84 , srv1310229727:84 , srv1320480761:84 , srv1323432398:84 , srv1324854569:84 , srv1347366285:84 , srv1400688147:84 , srv14090575:84 , srv1439115864:84 , srv145143559:84 , srv1457187693:84 , srv1479646910:84 , srv149434220:84 , srv149501089:84 , srv1500188257:84 , srv1501172592:84 , srv1503403731:84 , srv151007436:84 , srv1517759106:84 , srv1519486002:84 , srv1562648030:84 , srv1566084960:84 , srv1586729801:84 , srv1593847013:84 , srv167815320:84 , srv1747863160:84 , srv1760787090:84 , srv1765700281:84 , srv1784330465:84 , srv1785216271:84 , srv1819906108:84 , srv1883527985:84 , srv1902566779:84 , srv1934562425:84 , srv1967278410:84 , srv2002017870:84 , srv2019008812:84 , srv2031202489:84 , srv2037219585:84 , srv2047998499:84 , srv2100903941:84 , srv2103414392:84 , srv2126278899:84 , srv227755016:84 , srv236736249:84 , srv240421076:84 , srv244823165:84 , srv297566375:84 , srv314579593:84 , srv318669559:84 , srv373067761:84 , srv392598623:84 , srv410625740:84 , srv431312734:84 , srv432224950:84 , srv497826757:84 , srv513087656:84 , srv520983892:84 , srv563443839:84 , srv588161812:84 , srv590670904:84 , srv600222117:84 , srv62048464:84 , srv633015578:84 , srv662631620:84 , srv682825857:84 , srv725422746:84 , srv751418506:84 , srv762643747:84 , srv839434124:84 , srv840542558:84 , srv842906130:84 , srv852539944:84 , srv863375372:84 , srv877417584:84 , srv895759392:84 , srv905145250:84 , srv906131271:84 , srv906137311:84 , srv923504839:84 , srv92817717:84 , srv942298287:84 , srv966879716:84 , srv987054097:84 , srv619170999:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-05-12 20:53:24,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-12 20:53:24,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1562648030=35, srv1902566779=47, srv240421076=60, srv1008975073=1, srv1270681259=12, srv751418506=83, srv167815320=39, srv149434220=27, srv62048464=78, srv842906130=87, srv840542558=86, srv1152405931=6, srv2019008812=51, srv513087656=71, srv392598623=66, srv905145250=92, srv432224950=69, srv590670904=75, srv1255890433=10, srv725422746=82, srv762643747=84, srv1883527985=46, srv314579593=63, srv2100903941=55, srv662631620=80, srv2126278899=57, srv236736249=59, srv619170999=77, srv227755016=58, srv318669559=64, srv1765700281=42, srv1038764834=2, srv1400688147=21, srv1967278410=49, srv151007436=32, srv966879716=98, srv852539944=88, srv942298287=97, srv987054097=99, srv1172937503=7, srv1457187693=25, srv1199866789=8, srv1347366285=20, srv1784330465=43, srv923504839=95, srv1306662132=15, srv1760787090=41, srv1323432398=18, srv1503403731=31, srv600222117=76, srv2103414392=56, srv863375372=89, srv2002017870=50, srv906131271=93, srv682825857=81, srv1479646910=26, srv1593847013=38, srv877417584=90, srv145143559=24, srv1215282646=9, srv431312734=68, srv149501089=28, srv563443839=73, srv1519486002=34, srv2037219585=53, srv2047998499=54, srv497826757=70, srv1122827778=4, srv1586729801=37, srv839434124=85, srv1320480761=17, srv588161812=74, srv92817717=96, srv1785216271=44, srv373067761=65, srv1271521098=13, srv297566375=62, srv906137311=94, srv1747863160=40, srv1130896070=5, srv1259112757=11, srv1500188257=29, srv633015578=79, srv1007949230=0, srv1310229727=16, srv1439115864=23, srv2031202489=52, srv1284709569=14, srv1566084960=36, srv1819906108=45, srv1934562425=48, srv895759392=91, srv1517759106=33, srv244823165=61, srv1324854569=19, srv520983892=72, srv1055231135=3, srv14090575=22, srv1501172592=30, srv410625740=67} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-12 20:53:25,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-12 20:53:25,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-12 20:53:25,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-12 20:53:25,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-12 20:53:25,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-12 20:53:25,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-12 20:53:25,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-12 20:53:25,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-12 20:53:25,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-12 20:53:25,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-12 20:53:25,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-12 20:53:25,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-12 20:53:25,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-12 20:53:25,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-12 20:53:25,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-12 20:53:25,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-12 20:53:25,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-12 20:53:25,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-12 20:53:25,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-12 20:53:25,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-12 20:53:25,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-12 20:53:25,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-12 20:53:25,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-12 20:53:25,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-12 20:53:25,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-12 20:53:25,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-12 20:53:25,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-12 20:53:25,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-12 20:53:25,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-12 20:53:25,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-12 20:53:25,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-12 20:53:25,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-12 20:53:25,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-12 20:53:25,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-12 20:53:25,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-12 20:53:25,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-12 20:53:25,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-12 20:53:25,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-12 20:53:25,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-12 20:53:25,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-12 20:53:25,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-12 20:53:25,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-12 20:53:25,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-12 20:53:25,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-12 20:53:25,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-12 20:53:25,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-12 20:53:25,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-12 20:53:25,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-12 20:53:25,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-12 20:53:25,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-12 20:53:25,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-12 20:53:25,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-12 20:53:25,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-12 20:53:25,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-12 20:53:25,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-12 20:53:25,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-12 20:53:25,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-12 20:53:25,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-12 20:53:25,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-12 20:53:25,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-12 20:53:25,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-12 20:53:25,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-12 20:53:25,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-12 20:53:25,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-12 20:53:25,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-12 20:53:25,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-12 20:53:25,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-12 20:53:25,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-12 20:53:25,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-12 20:53:25,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-12 20:53:25,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-12 20:53:25,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-12 20:53:25,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-12 20:53:25,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-12 20:53:25,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-12 20:53:25,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-12 20:53:25,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-12 20:53:25,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-12 20:53:25,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-12 20:53:25,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-12 20:53:25,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-12 20:53:25,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-12 20:53:25,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-12 20:53:25,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-12 20:53:25,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-12 20:53:25,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-12 20:53:25,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-12 20:53:25,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-12 20:53:25,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-12 20:53:25,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-12 20:53:25,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-12 20:53:25,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-12 20:53:25,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-12 20:53:25,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-12 20:53:25,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-12 20:53:25,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-12 20:53:25,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-12 20:53:25,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-12 20:53:25,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-12 20:53:25,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-12 20:53:25,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-12 20:53:25,516 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-12 20:53:25,517 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-12 20:53:25,517 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-12 20:53:26,265 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 1124 ms to try 5000 different iterations.  Found a solution that moves 4729 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1796209278111326. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03683750968734291, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05506857269999256, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5254444444444445, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03683750968734291, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17483333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24175, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-12 20:53:26,287 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1008975073:82 , srv1259112757:82 , srv1400688147:82 , srv1784330465:82 , srv2019008812:82 , srv236736249:82 , srv431312734:83 , srv1055231135:84 , srv1271521098:84 , srv1347366285:84 , srv1479646910:84 , srv149501089:84 , srv1501172592:84 , srv1503403731:84 , srv167815320:84 , srv2047998499:84 , srv2126278899:84 , srv314579593:84 , srv318669559:84 , srv682825857:84 , srv842906130:84 , srv942298287:84 , srv987054097:84 , srv1038764834:85 , srv1130896070:85 , srv1439115864:85 , srv1566084960:85 , srv1819906108:85 , srv2100903941:85 , srv392598623:85 , srv410625740:85 , srv563443839:85 , srv600222117:85 , srv895759392:85 , srv905145250:85 , srv1007949230:86 , srv1172937503:86 , srv1199866789:86 , srv1284709569:86 , srv1323432398:86 , srv14090575:86 , srv151007436:86 , srv1562648030:86 , srv1785216271:86 , srv227755016:86 , srv240421076:86 , srv62048464:86 , srv762643747:86 , srv1122827778:87 , srv1255890433:87 , srv1306662132:87 , srv1310229727:87 , srv1320480761:87 , srv145143559:87 , srv149434220:87 , srv1747863160:87 , srv1902566779:87 , srv1967278410:87 , srv2031202489:87 , srv2103414392:87 , srv244823165:87 , srv297566375:87 , srv432224950:87 , srv513087656:87 , srv520983892:87 , srv588161812:87 , srv633015578:87 , srv839434124:87 , srv852539944:87 , srv1215282646:88 , srv1270681259:88 , srv1324854569:88 , srv1760787090:88 , srv1883527985:88 , srv2002017870:88 , srv373067761:88 , srv590670904:88 , srv751418506:88 , srv906131271:88 , srv923504839:88 , srv1500188257:89 , srv1517759106:89 , srv1765700281:89 , srv1934562425:89 , srv2037219585:89 , srv863375372:89 , srv877417584:89 , srv92817717:89 , srv662631620:90 , srv906137311:90 , srv1152405931:91 , srv497826757:91 , srv1519486002:92 , srv1586729801:92 , srv725422746:92 , srv1457187693:94 , srv840542558:94 , srv966879716:96 , srv1593847013:97 , srv619170999:417 }
2023-05-12 20:53:26,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-12 20:53:26,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1562648030=35, srv1902566779=47, srv240421076=60, srv1008975073=1, srv1270681259=12, srv751418506=83, srv167815320=39, srv149434220=27, srv62048464=78, srv842906130=87, srv840542558=86, srv1152405931=6, srv2019008812=51, srv513087656=71, srv392598623=66, srv905145250=92, srv432224950=69, srv590670904=75, srv1255890433=10, srv725422746=82, srv762643747=84, srv1883527985=46, srv314579593=63, srv2100903941=55, srv662631620=80, srv2126278899=57, srv236736249=59, srv619170999=77, srv227755016=58, srv318669559=64, srv1765700281=42, srv1038764834=2, srv1400688147=21, srv1967278410=49, srv151007436=32, srv966879716=98, srv852539944=88, srv942298287=97, srv987054097=99, srv1172937503=7, srv1457187693=25, srv1199866789=8, srv1347366285=20, srv1784330465=43, srv923504839=95, srv1306662132=15, srv1760787090=41, srv1323432398=18, srv1503403731=31, srv600222117=76, srv2103414392=56, srv863375372=89, srv2002017870=50, srv906131271=93, srv682825857=81, srv1479646910=26, srv1593847013=38, srv877417584=90, srv145143559=24, srv1215282646=9, srv431312734=68, srv149501089=28, srv563443839=73, srv1519486002=34, srv2037219585=53, srv2047998499=54, srv497826757=70, srv1122827778=4, srv1586729801=37, srv839434124=85, srv1320480761=17, srv588161812=74, srv92817717=96, srv1785216271=44, srv373067761=65, srv1271521098=13, srv297566375=62, srv906137311=94, srv1747863160=40, srv1130896070=5, srv1259112757=11, srv1500188257=29, srv633015578=79, srv1007949230=0, srv1310229727=16, srv1439115864=23, srv2031202489=52, srv1284709569=14, srv1566084960=36, srv1819906108=45, srv1934562425=48, srv895759392=91, srv1517759106=33, srv244823165=61, srv1324854569=19, srv520983892=72, srv1055231135=3, srv14090575=22, srv1501172592=30, srv410625740=67} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-12 20:53:26,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-12 20:53:26,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-12 20:53:26,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-12 20:53:26,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-12 20:53:26,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-12 20:53:26,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-12 20:53:26,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-12 20:53:26,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-12 20:53:26,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-12 20:53:26,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-12 20:53:26,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-12 20:53:26,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-12 20:53:26,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-12 20:53:26,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-12 20:53:26,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-12 20:53:26,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-12 20:53:26,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-12 20:53:26,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-12 20:53:26,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-12 20:53:26,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-12 20:53:26,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-12 20:53:26,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-12 20:53:26,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-12 20:53:26,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-12 20:53:26,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-12 20:53:26,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-12 20:53:26,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-12 20:53:26,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-12 20:53:26,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-12 20:53:26,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-12 20:53:26,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-12 20:53:26,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-12 20:53:26,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-12 20:53:26,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-12 20:53:26,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-12 20:53:26,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-12 20:53:26,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-12 20:53:26,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-12 20:53:26,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-12 20:53:26,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-12 20:53:26,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-12 20:53:26,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-12 20:53:26,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-12 20:53:26,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-12 20:53:26,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-12 20:53:26,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-12 20:53:26,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-12 20:53:26,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-12 20:53:26,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-12 20:53:26,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-12 20:53:26,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-12 20:53:26,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-12 20:53:26,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-12 20:53:26,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-12 20:53:26,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-12 20:53:26,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-12 20:53:26,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-12 20:53:26,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-12 20:53:26,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-12 20:53:26,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-12 20:53:26,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-12 20:53:26,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-12 20:53:26,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-12 20:53:26,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-12 20:53:26,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-12 20:53:26,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-12 20:53:26,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-12 20:53:26,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-12 20:53:26,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-12 20:53:26,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-12 20:53:26,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-12 20:53:26,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-12 20:53:26,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-12 20:53:26,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-12 20:53:26,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-12 20:53:26,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-12 20:53:26,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-12 20:53:26,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-12 20:53:26,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-12 20:53:26,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-12 20:53:26,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-12 20:53:26,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-12 20:53:26,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-12 20:53:26,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-12 20:53:26,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-12 20:53:26,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-12 20:53:26,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-12 20:53:26,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-12 20:53:26,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-12 20:53:26,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-12 20:53:26,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-12 20:53:26,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-12 20:53:26,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-12 20:53:26,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-12 20:53:26,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-12 20:53:26,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-12 20:53:26,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-12 20:53:26,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-12 20:53:26,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-12 20:53:26,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-12 20:53:26,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-12 20:53:26,532 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03683750968734291, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05506857269999256, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03683750968734291, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17483333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24175, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-12 20:53:26,533 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-12 20:53:26,533 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.1795878146453907, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03683750968734291, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05506857269999256, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03683750968734291, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17483333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24175, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-12 20:53:26,857 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 491 ms to try 5000 different iterations.  Found a solution that moves 2624 regions; Going from a computed imbalance of 0.1795878146453907 to a new imbalance of 0.014686928256828142. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.019920655680937024); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.032362980611018825, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29155555555555557, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.019920655680937024); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011833333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.041916666666666665, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-12 20:53:26,868 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv966879716:75 , srv1593847013:79 , srv840542558:79 , srv1152405931:80 , srv1457187693:80 , srv497826757:80 , srv725422746:80 , srv1586729801:81 , srv1517759106:82 , srv863375372:82 , srv1320480761:83 , srv1324854569:83 , srv1519486002:83 , srv1883527985:83 , srv1934562425:83 , srv2037219585:83 , srv590670904:83 , srv662631620:83 , srv877417584:83 , srv906137311:83 , srv92817717:83 , srv1270681259:84 , srv145143559:84 , srv1760787090:84 , srv1902566779:84 , srv2002017870:84 , srv2031202489:84 , srv373067761:84 , srv513087656:84 , srv751418506:84 , srv1122827778:85 , srv1130896070:85 , srv1172937503:85 , srv1284709569:85 , srv1306662132:85 , srv1310229727:85 , srv1323432398:85 , srv1500188257:85 , srv1747863160:85 , srv1967278410:85 , srv240421076:85 , srv244823165:85 , srv432224950:85 , srv588161812:85 , srv600222117:85 , srv62048464:85 , srv852539944:85 , srv905145250:85 , srv906131271:85 , srv923504839:85 , srv1215282646:86 , srv1255890433:86 , srv1271521098:86 , srv1347366285:86 , srv14090575:86 , srv1439115864:86 , srv1501172592:86 , srv1566084960:86 , srv1765700281:86 , srv2047998499:86 , srv227755016:86 , srv297566375:86 , srv633015578:86 , srv839434124:86 , srv1199866789:87 , srv151007436:87 , srv1785216271:87 , srv1819906108:87 , srv2103414392:87 , srv392598623:87 , srv410625740:87 , srv520983892:87 , srv563443839:87 , srv762643747:87 , srv1007949230:88 , srv1038764834:88 , srv1055231135:88 , srv149434220:88 , srv149501089:88 , srv167815320:88 , srv1784330465:88 , srv2100903941:88 , srv318669559:88 , srv431312734:88 , srv682825857:88 , srv942298287:88 , srv1008975073:89 , srv1479646910:89 , srv1503403731:89 , srv1562648030:89 , srv314579593:89 , srv895759392:89 , srv1400688147:90 , srv842906130:90 , srv1259112757:91 , srv2019008812:91 , srv2126278899:91 , srv236736249:91 , srv987054097:91 , srv619170999:534 }
2023-05-12 20:53:26,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-12 20:53:26,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1562648030=35, srv1902566779=47, srv240421076=60, srv1008975073=1, srv1270681259=12, srv751418506=83, srv167815320=39, srv149434220=27, srv62048464=78, srv842906130=87, srv840542558=86, srv1152405931=6, srv2019008812=51, srv513087656=71, srv392598623=66, srv905145250=92, srv432224950=69, srv590670904=75, srv1255890433=10, srv725422746=82, srv762643747=84, srv1883527985=46, srv314579593=63, srv2100903941=55, srv662631620=80, srv2126278899=57, srv236736249=59, srv619170999=77, srv227755016=58, srv318669559=64, srv1765700281=42, srv1038764834=2, srv1400688147=21, srv1967278410=49, srv151007436=32, srv966879716=98, srv852539944=88, srv942298287=97, srv987054097=99, srv1172937503=7, srv1457187693=25, srv1199866789=8, srv1347366285=20, srv1784330465=43, srv923504839=95, srv1306662132=15, srv1760787090=41, srv1323432398=18, srv1503403731=31, srv600222117=76, srv2103414392=56, srv863375372=89, srv2002017870=50, srv906131271=93, srv682825857=81, srv1479646910=26, srv1593847013=38, srv877417584=90, srv145143559=24, srv1215282646=9, srv431312734=68, srv149501089=28, srv563443839=73, srv1519486002=34, srv2037219585=53, srv2047998499=54, srv497826757=70, srv1122827778=4, srv1586729801=37, srv839434124=85, srv1320480761=17, srv588161812=74, srv92817717=96, srv1785216271=44, srv373067761=65, srv1271521098=13, srv297566375=62, srv906137311=94, srv1747863160=40, srv1130896070=5, srv1259112757=11, srv1500188257=29, srv633015578=79, srv1007949230=0, srv1310229727=16, srv1439115864=23, srv2031202489=52, srv1284709569=14, srv1566084960=36, srv1819906108=45, srv1934562425=48, srv895759392=91, srv1517759106=33, srv244823165=61, srv1324854569=19, srv520983892=72, srv1055231135=3, srv14090575=22, srv1501172592=30, srv410625740=67} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-12 20:53:26,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-12 20:53:26,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-12 20:53:26,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-12 20:53:26,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-12 20:53:26,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-12 20:53:26,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-12 20:53:26,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-12 20:53:26,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-12 20:53:26,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-12 20:53:26,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-12 20:53:26,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-12 20:53:26,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-12 20:53:26,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-12 20:53:26,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-12 20:53:26,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-12 20:53:26,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-12 20:53:26,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-12 20:53:26,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-12 20:53:26,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-12 20:53:26,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-12 20:53:26,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-12 20:53:26,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-12 20:53:26,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-12 20:53:26,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-12 20:53:26,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-12 20:53:26,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-12 20:53:26,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-12 20:53:26,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-12 20:53:26,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-12 20:53:26,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-12 20:53:26,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-12 20:53:26,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-12 20:53:26,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-12 20:53:26,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-12 20:53:26,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-12 20:53:26,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-12 20:53:26,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-12 20:53:26,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-12 20:53:26,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-12 20:53:26,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-12 20:53:26,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-12 20:53:26,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-12 20:53:26,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-12 20:53:26,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-12 20:53:26,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-12 20:53:26,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-12 20:53:26,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-12 20:53:26,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-12 20:53:26,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-12 20:53:26,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-12 20:53:26,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-12 20:53:26,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-12 20:53:26,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-12 20:53:26,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-12 20:53:26,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-12 20:53:26,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-12 20:53:26,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-12 20:53:26,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-12 20:53:26,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-12 20:53:26,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-12 20:53:26,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-12 20:53:26,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-12 20:53:26,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-12 20:53:26,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-12 20:53:26,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-12 20:53:26,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-12 20:53:26,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-12 20:53:26,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-12 20:53:26,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-12 20:53:26,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-12 20:53:26,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-12 20:53:26,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-12 20:53:26,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-12 20:53:26,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-12 20:53:26,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-12 20:53:26,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-12 20:53:26,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-12 20:53:26,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-12 20:53:26,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-12 20:53:26,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-12 20:53:26,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-12 20:53:26,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-12 20:53:26,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-12 20:53:26,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-12 20:53:26,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-12 20:53:26,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-12 20:53:26,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-12 20:53:26,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-12 20:53:26,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-12 20:53:26,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-12 20:53:26,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-12 20:53:26,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-12 20:53:26,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-12 20:53:26,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-12 20:53:26,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-12 20:53:26,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-12 20:53:26,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-12 20:53:26,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-12 20:53:26,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-12 20:53:26,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-12 20:53:26,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-12 20:53:27,061 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.019920655680937024); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.032362980611018825, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.019920655680937024); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011833333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.041916666666666665, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-12 20:53:27,062 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-12 20:53:27,062 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.014668554616120443, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.019920655680937024); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.032362980611018825, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.019920655680937024); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011833333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.041916666666666665, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-12 20:53:27,384 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 449 ms to try 5000 different iterations.  Found a solution that moves 1007 regions; Going from a computed imbalance of 0.014668554616120443 to a new imbalance of 3.5374674281884116E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0049715489327143165); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00670025210172808); MoveCostFunction : (multiplier=7.0, imbalance=0.11188888888888888, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0049715489327143165); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.002416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-12 20:53:27,386 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv149434220:82 , srv1593847013:82 , srv2126278899:82 , srv1457187693:83 , srv1519486002:83 , srv1562648030:83 , srv1765700281:83 , srv520983892:83 , srv895759392:83 , srv987054097:83 , srv1007949230:84 , srv1038764834:84 , srv1215282646:84 , srv1255890433:84 , srv1479646910:84 , srv1500188257:84 , srv1503403731:84 , srv151007436:84 , srv1586729801:84 , srv2103414392:84 , srv236736249:84 , srv314579593:84 , srv633015578:84 , srv762643747:84 , srv839434124:84 , srv842906130:84 , srv906131271:84 , srv923504839:84 , srv1055231135:85 , srv1122827778:85 , srv1199866789:85 , srv1259112757:85 , srv1310229727:85 , srv1400688147:85 , srv149501089:85 , srv1785216271:85 , srv1819906108:85 , srv2019008812:85 , srv2100903941:85 , srv244823165:85 , srv297566375:85 , srv373067761:85 , srv410625740:85 , srv588161812:85 , srv662631620:85 , srv751418506:85 , srv840542558:85 , srv877417584:85 , srv906137311:85 , srv92817717:85 , srv942298287:85 , srv1008975073:86 , srv1172937503:86 , srv1270681259:86 , srv1284709569:86 , srv1306662132:86 , srv14090575:86 , srv145143559:86 , srv1517759106:86 , srv1566084960:86 , srv167815320:86 , srv1747863160:86 , srv1760787090:86 , srv1883527985:86 , srv1934562425:86 , srv1967278410:86 , srv2002017870:86 , srv2031202489:86 , srv2037219585:86 , srv227755016:86 , srv318669559:86 , srv392598623:86 , srv432224950:86 , srv497826757:86 , srv563443839:86 , srv590670904:86 , srv682825857:86 , srv725422746:86 , srv852539944:86 , srv1130896070:87 , srv1152405931:87 , srv1320480761:87 , srv1323432398:87 , srv1324854569:87 , srv1439115864:87 , srv1784330465:87 , srv1902566779:87 , srv240421076:87 , srv513087656:87 , srv62048464:87 , srv863375372:87 , srv966879716:87 , srv1271521098:88 , srv1347366285:88 , srv1501172592:88 , srv2047998499:88 , srv431312734:88 , srv600222117:88 , srv905145250:88 , srv619170999:551 }
2023-05-12 20:53:27,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-12 20:53:27,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1562648030=35, srv1902566779=47, srv240421076=60, srv1008975073=1, srv1270681259=12, srv751418506=83, srv167815320=39, srv149434220=27, srv62048464=78, srv842906130=87, srv840542558=86, srv1152405931=6, srv2019008812=51, srv513087656=71, srv392598623=66, srv905145250=92, srv432224950=69, srv590670904=75, srv1255890433=10, srv725422746=82, srv762643747=84, srv1883527985=46, srv314579593=63, srv2100903941=55, srv662631620=80, srv2126278899=57, srv236736249=59, srv619170999=77, srv227755016=58, srv318669559=64, srv1765700281=42, srv1038764834=2, srv1400688147=21, srv1967278410=49, srv151007436=32, srv966879716=98, srv852539944=88, srv942298287=97, srv987054097=99, srv1172937503=7, srv1457187693=25, srv1199866789=8, srv1347366285=20, srv1784330465=43, srv923504839=95, srv1306662132=15, srv1760787090=41, srv1323432398=18, srv1503403731=31, srv600222117=76, srv2103414392=56, srv863375372=89, srv2002017870=50, srv906131271=93, srv682825857=81, srv1479646910=26, srv1593847013=38, srv877417584=90, srv145143559=24, srv1215282646=9, srv431312734=68, srv149501089=28, srv563443839=73, srv1519486002=34, srv2037219585=53, srv2047998499=54, srv497826757=70, srv1122827778=4, srv1586729801=37, srv839434124=85, srv1320480761=17, srv588161812=74, srv92817717=96, srv1785216271=44, srv373067761=65, srv1271521098=13, srv297566375=62, srv906137311=94, srv1747863160=40, srv1130896070=5, srv1259112757=11, srv1500188257=29, srv633015578=79, srv1007949230=0, srv1310229727=16, srv1439115864=23, srv2031202489=52, srv1284709569=14, srv1566084960=36, srv1819906108=45, srv1934562425=48, srv895759392=91, srv1517759106=33, srv244823165=61, srv1324854569=19, srv520983892=72, srv1055231135=3, srv14090575=22, srv1501172592=30, srv410625740=67} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-12 20:53:27,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-12 20:53:27,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-12 20:53:27,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-12 20:53:27,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-12 20:53:27,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-12 20:53:27,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-12 20:53:27,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-12 20:53:27,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-12 20:53:27,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-12 20:53:27,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-12 20:53:27,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-12 20:53:27,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-12 20:53:27,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-12 20:53:27,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-12 20:53:27,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-12 20:53:27,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-12 20:53:27,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-12 20:53:27,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-12 20:53:27,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-12 20:53:27,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-12 20:53:27,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-12 20:53:27,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-12 20:53:27,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-12 20:53:27,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-12 20:53:27,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-12 20:53:27,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-12 20:53:27,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-12 20:53:27,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-12 20:53:27,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-12 20:53:27,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-12 20:53:27,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-12 20:53:27,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-12 20:53:27,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-12 20:53:27,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-12 20:53:27,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-12 20:53:27,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-12 20:53:27,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-12 20:53:27,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-12 20:53:27,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-12 20:53:27,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-12 20:53:27,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-12 20:53:27,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-12 20:53:27,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-12 20:53:27,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-12 20:53:27,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-12 20:53:27,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-12 20:53:27,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-12 20:53:27,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-12 20:53:27,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-12 20:53:27,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-12 20:53:27,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-12 20:53:27,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-12 20:53:27,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-12 20:53:27,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-12 20:53:27,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-12 20:53:27,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-12 20:53:27,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-12 20:53:27,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-12 20:53:27,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-12 20:53:27,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-12 20:53:27,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-12 20:53:27,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-12 20:53:27,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-12 20:53:27,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-12 20:53:27,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-12 20:53:27,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-12 20:53:27,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-12 20:53:27,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-12 20:53:27,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-12 20:53:27,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-12 20:53:27,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-12 20:53:27,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-12 20:53:27,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-12 20:53:27,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-12 20:53:27,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-12 20:53:27,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-12 20:53:27,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-12 20:53:27,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-12 20:53:27,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-12 20:53:27,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-12 20:53:27,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-12 20:53:27,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-12 20:53:27,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-12 20:53:27,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-12 20:53:27,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-12 20:53:27,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-12 20:53:27,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-12 20:53:27,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-12 20:53:27,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-12 20:53:27,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-12 20:53:27,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-12 20:53:27,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-12 20:53:27,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-12 20:53:27,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-12 20:53:27,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-12 20:53:27,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-12 20:53:27,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-12 20:53:27,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-12 20:53:27,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-12 20:53:27,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-12 20:53:27,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-12 20:53:27,534 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0049715489327143165); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00670025210172808); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0049715489327143165); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.002416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-12 20:53:27,534 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-12 20:53:27,535 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=3.46695578111276E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0049715489327143165); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00670025210172808); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0049715489327143165); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.002416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-12 20:53:27,870 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 437 ms to try 5000 different iterations.  Found a solution that moves 206 regions; Going from a computed imbalance of 3.46695578111276E-4 to a new imbalance of 1.0531064112930606E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.5313429231318235E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0016412198797244363); MoveCostFunction : (multiplier=7.0, imbalance=0.02288888888888889); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.5313429231318235E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-12 20:53:27,871 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1215282646:83 , srv1271521098:83 , srv431312734:83 , srv863375372:83 , srv1007949230:84 , srv1152405931:84 , srv1199866789:84 , srv1259112757:84 , srv1306662132:84 , srv1323432398:84 , srv1324854569:84 , srv1347366285:84 , srv1400688147:84 , srv14090575:84 , srv1439115864:84 , srv1457187693:84 , srv1500188257:84 , srv1501172592:84 , srv1519486002:84 , srv1562648030:84 , srv1593847013:84 , srv167815320:84 , srv1747863160:84 , srv1760787090:84 , srv1765700281:84 , srv1785216271:84 , srv1902566779:84 , srv1934562425:84 , srv1967278410:84 , srv2002017870:84 , srv2019008812:84 , srv2037219585:84 , srv2047998499:84 , srv2100903941:84 , srv2103414392:84 , srv227755016:84 , srv240421076:84 , srv297566375:84 , srv318669559:84 , srv432224950:84 , srv563443839:84 , srv600222117:84 , srv62048464:84 , srv662631620:84 , srv682825857:84 , srv725422746:84 , srv840542558:84 , srv842906130:84 , srv852539944:84 , srv905145250:84 , srv906137311:84 , srv966879716:84 , srv987054097:84 , srv1008975073:85 , srv1038764834:85 , srv1055231135:85 , srv1122827778:85 , srv1130896070:85 , srv1255890433:85 , srv1270681259:85 , srv1284709569:85 , srv1310229727:85 , srv1320480761:85 , srv145143559:85 , srv1479646910:85 , srv149434220:85 , srv149501089:85 , srv1503403731:85 , srv151007436:85 , srv1517759106:85 , srv1566084960:85 , srv1586729801:85 , srv1784330465:85 , srv1819906108:85 , srv2031202489:85 , srv2126278899:85 , srv236736249:85 , srv244823165:85 , srv314579593:85 , srv373067761:85 , srv392598623:85 , srv410625740:85 , srv497826757:85 , srv513087656:85 , srv520983892:85 , srv588161812:85 , srv590670904:85 , srv633015578:85 , srv751418506:85 , srv762643747:85 , srv839434124:85 , srv877417584:85 , srv895759392:85 , srv906131271:85 , srv923504839:85 , srv92817717:85 , srv942298287:85 , srv1172937503:86 , srv1883527985:86 , srv619170999:640 }
2023-05-12 20:53:27,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-12 20:53:27,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1562648030=35, srv1902566779=47, srv240421076=60, srv1008975073=1, srv1270681259=12, srv751418506=83, srv167815320=39, srv149434220=27, srv62048464=78, srv842906130=87, srv840542558=86, srv1152405931=6, srv2019008812=51, srv513087656=71, srv392598623=66, srv905145250=92, srv432224950=69, srv590670904=75, srv1255890433=10, srv725422746=82, srv762643747=84, srv1883527985=46, srv314579593=63, srv2100903941=55, srv662631620=80, srv2126278899=57, srv236736249=59, srv619170999=77, srv227755016=58, srv318669559=64, srv1765700281=42, srv1038764834=2, srv1400688147=21, srv1967278410=49, srv151007436=32, srv966879716=98, srv852539944=88, srv942298287=97, srv987054097=99, srv1172937503=7, srv1457187693=25, srv1199866789=8, srv1347366285=20, srv1784330465=43, srv923504839=95, srv1306662132=15, srv1760787090=41, srv1323432398=18, srv1503403731=31, srv600222117=76, srv2103414392=56, srv863375372=89, srv2002017870=50, srv906131271=93, srv682825857=81, srv1479646910=26, srv1593847013=38, srv877417584=90, srv145143559=24, srv1215282646=9, srv431312734=68, srv149501089=28, srv563443839=73, srv1519486002=34, srv2037219585=53, srv2047998499=54, srv497826757=70, srv1122827778=4, srv1586729801=37, srv839434124=85, srv1320480761=17, srv588161812=74, srv92817717=96, srv1785216271=44, srv373067761=65, srv1271521098=13, srv297566375=62, srv906137311=94, srv1747863160=40, srv1130896070=5, srv1259112757=11, srv1500188257=29, srv633015578=79, srv1007949230=0, srv1310229727=16, srv1439115864=23, srv2031202489=52, srv1284709569=14, srv1566084960=36, srv1819906108=45, srv1934562425=48, srv895759392=91, srv1517759106=33, srv244823165=61, srv1324854569=19, srv520983892=72, srv1055231135=3, srv14090575=22, srv1501172592=30, srv410625740=67} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-12 20:53:27,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-12 20:53:27,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-12 20:53:27,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-12 20:53:27,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-12 20:53:27,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-12 20:53:27,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-12 20:53:27,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-12 20:53:27,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-12 20:53:27,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-12 20:53:27,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-12 20:53:27,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-12 20:53:27,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-12 20:53:27,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-12 20:53:27,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-12 20:53:27,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-12 20:53:27,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-12 20:53:27,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-12 20:53:27,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-12 20:53:27,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-12 20:53:27,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-12 20:53:27,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-12 20:53:27,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-12 20:53:27,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-12 20:53:27,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-12 20:53:27,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-12 20:53:27,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-12 20:53:27,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-12 20:53:27,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-12 20:53:27,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-12 20:53:27,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-12 20:53:27,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-12 20:53:27,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-12 20:53:27,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-12 20:53:27,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-12 20:53:27,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-12 20:53:27,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-12 20:53:27,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-12 20:53:27,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-12 20:53:27,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-12 20:53:27,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-12 20:53:27,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-12 20:53:27,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-12 20:53:27,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-12 20:53:27,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-12 20:53:27,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-12 20:53:27,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-12 20:53:27,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-12 20:53:27,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-12 20:53:27,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-12 20:53:27,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-12 20:53:27,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-12 20:53:27,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-12 20:53:27,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-12 20:53:27,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-12 20:53:27,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-12 20:53:27,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-12 20:53:27,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-12 20:53:27,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-12 20:53:27,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-12 20:53:27,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-12 20:53:27,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-12 20:53:27,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-12 20:53:27,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-12 20:53:27,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-12 20:53:27,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-12 20:53:27,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-12 20:53:27,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-12 20:53:27,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-12 20:53:27,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-12 20:53:27,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-12 20:53:27,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-12 20:53:27,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-12 20:53:27,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-12 20:53:27,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-12 20:53:27,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-12 20:53:27,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-12 20:53:27,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-12 20:53:27,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-12 20:53:27,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-12 20:53:27,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-12 20:53:27,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-12 20:53:27,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-12 20:53:27,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-12 20:53:27,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-12 20:53:27,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-12 20:53:27,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-12 20:53:27,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-12 20:53:27,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-12 20:53:27,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-12 20:53:27,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-12 20:53:27,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-12 20:53:27,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-12 20:53:27,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-12 20:53:27,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-12 20:53:27,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-12 20:53:27,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-12 20:53:27,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-12 20:53:27,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-12 20:53:27,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-12 20:53:27,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-12 20:53:27,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-12 20:53:28,016 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=9.08862128298181E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.5313429231318235E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0016412198797244363); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.5313429231318235E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-12 20:53:28,016 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1215282646:83 , srv1271521098:83 , srv431312734:83 , srv863375372:83 , srv1007949230:84 , srv1152405931:84 , srv1199866789:84 , srv1259112757:84 , srv1306662132:84 , srv1323432398:84 , srv1324854569:84 , srv1347366285:84 , srv1400688147:84 , srv14090575:84 , srv1439115864:84 , srv1457187693:84 , srv1500188257:84 , srv1501172592:84 , srv1519486002:84 , srv1562648030:84 , srv1593847013:84 , srv167815320:84 , srv1747863160:84 , srv1760787090:84 , srv1765700281:84 , srv1785216271:84 , srv1902566779:84 , srv1934562425:84 , srv1967278410:84 , srv2002017870:84 , srv2019008812:84 , srv2037219585:84 , srv2047998499:84 , srv2100903941:84 , srv2103414392:84 , srv227755016:84 , srv240421076:84 , srv297566375:84 , srv318669559:84 , srv432224950:84 , srv563443839:84 , srv600222117:84 , srv62048464:84 , srv662631620:84 , srv682825857:84 , srv725422746:84 , srv840542558:84 , srv842906130:84 , srv852539944:84 , srv905145250:84 , srv906137311:84 , srv966879716:84 , srv987054097:84 , srv1008975073:85 , srv1038764834:85 , srv1055231135:85 , srv1122827778:85 , srv1130896070:85 , srv1255890433:85 , srv1270681259:85 , srv1284709569:85 , srv1310229727:85 , srv1320480761:85 , srv145143559:85 , srv1479646910:85 , srv149434220:85 , srv149501089:85 , srv1503403731:85 , srv151007436:85 , srv1517759106:85 , srv1566084960:85 , srv1586729801:85 , srv1784330465:85 , srv1819906108:85 , srv2031202489:85 , srv2126278899:85 , srv236736249:85 , srv244823165:85 , srv314579593:85 , srv373067761:85 , srv392598623:85 , srv410625740:85 , srv497826757:85 , srv513087656:85 , srv520983892:85 , srv588161812:85 , srv590670904:85 , srv633015578:85 , srv751418506:85 , srv762643747:85 , srv839434124:85 , srv877417584:85 , srv895759392:85 , srv906131271:85 , srv923504839:85 , srv92817717:85 , srv942298287:85 , srv1172937503:86 , srv1883527985:86 , srv619170999:640 }
2023-05-12 20:53:28,044 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=345 (was 345), ProcessCount=171 (was 171), AvailableMemoryMB=9494 (was 10354)
2023-05-12 20:53:28,054 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=345, ProcessCount=171, AvailableMemoryMB=9494
2023-05-12 20:53:28,054 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-12 20:53:28,055 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-05-12 20:53:28,067 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv109461224:84 , srv1094774575:84 , srv1210441996:84 , srv1217289640:84 , srv1328848809:84 , srv133802805:84 , srv1501032106:84 , srv1516107263:84 , srv1560514153:84 , srv1643611248:84 , srv1700190074:84 , srv1890723544:84 , srv1899167711:84 , srv1904689054:84 , srv1950260687:84 , srv1989401090:84 , srv1990199606:84 , srv225834847:84 , srv278101039:84 , srv343098669:84 , srv366892392:84 , srv425822825:84 , srv484249641:84 , srv513966694:84 , srv536712643:84 , srv756570198:84 , srv826030085:84 , srv939974148:84 , srv958003250:84 , srv931672278:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-05-12 20:53:28,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-12 20:53:28,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv484249641=22, srv343098669=19, srv366892392=20, srv1560514153=8, srv513966694=23, srv931672278=27, srv1501032106=6, srv1643611248=9, srv425822825=21, srv278101039=18, srv1328848809=4, srv1094774575=1, srv1516107263=7, srv109461224=0, srv1990199606=16, srv1890723544=11, srv225834847=17, srv939974148=28, srv1950260687=14, srv1899167711=12, srv133802805=5, srv1904689054=13, srv958003250=29, srv536712643=24, srv756570198=25, srv826030085=26, srv1989401090=15, srv1217289640=3, srv1700190074=10, srv1210441996=2} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-12 20:53:28,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-12 20:53:28,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-12 20:53:28,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-12 20:53:28,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-12 20:53:28,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-12 20:53:28,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-12 20:53:28,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-12 20:53:28,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-12 20:53:28,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-12 20:53:28,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-12 20:53:28,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-12 20:53:28,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-12 20:53:28,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-12 20:53:28,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-12 20:53:28,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-12 20:53:28,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-12 20:53:28,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-12 20:53:28,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-12 20:53:28,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-12 20:53:28,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-12 20:53:28,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-12 20:53:28,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-12 20:53:28,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-12 20:53:28,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-12 20:53:28,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-12 20:53:28,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-12 20:53:28,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-12 20:53:28,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-12 20:53:28,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-12 20:53:28,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-12 20:53:28,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-05-12 20:53:28,114 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0690621868051318, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-12 20:53:28,114 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912410655959816, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0690621868051318, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-05-12 20:55:28,090 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6826979 different iterations.  Found a solution that moves 1854 regions; Going from a computed imbalance of 0.9912410655959816 to a new imbalance of 4.3273284898463826E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6866666666666666, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-12 20:55:28,097 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv109461224:90 , srv1094774575:90 , srv1210441996:90 , srv1217289640:90 , srv1328848809:90 , srv133802805:90 , srv1501032106:90 , srv1516107263:90 , srv1560514153:90 , srv1643611248:90 , srv1700190074:90 , srv1890723544:90 , srv1899167711:90 , srv1904689054:90 , srv1950260687:90 , srv1989401090:90 , srv1990199606:90 , srv225834847:90 , srv278101039:90 , srv343098669:90 , srv366892392:90 , srv425822825:90 , srv484249641:90 , srv513966694:90 , srv536712643:90 , srv756570198:90 , srv826030085:90 , srv931672278:90 , srv939974148:90 , srv958003250:90 }
2023-05-12 20:55:28,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-12 20:55:28,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv484249641=22, srv343098669=19, srv366892392=20, srv1560514153=8, srv513966694=23, srv931672278=27, srv1501032106=6, srv1643611248=9, srv425822825=21, srv278101039=18, srv1328848809=4, srv1094774575=1, srv1516107263=7, srv109461224=0, srv1990199606=16, srv1890723544=11, srv225834847=17, srv939974148=28, srv1950260687=14, srv1899167711=12, srv133802805=5, srv1904689054=13, srv958003250=29, srv536712643=24, srv756570198=25, srv826030085=26, srv1989401090=15, srv1217289640=3, srv1700190074=10, srv1210441996=2} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-12 20:55:28,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-12 20:55:28,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-12 20:55:28,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-12 20:55:28,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-12 20:55:28,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-12 20:55:28,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-12 20:55:28,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-12 20:55:28,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-12 20:55:28,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-12 20:55:28,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-12 20:55:28,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-12 20:55:28,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-12 20:55:28,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-12 20:55:28,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-12 20:55:28,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-12 20:55:28,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-12 20:55:28,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-12 20:55:28,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-12 20:55:28,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-12 20:55:28,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-12 20:55:28,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-12 20:55:28,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-12 20:55:28,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-12 20:55:28,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-12 20:55:28,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-12 20:55:28,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-12 20:55:28,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-12 20:55:28,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-12 20:55:28,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-12 20:55:28,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-12 20:55:28,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-05-12 20:55:28,117 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-12 20:55:28,117 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv109461224:90 , srv1094774575:90 , srv1210441996:90 , srv1217289640:90 , srv1328848809:90 , srv133802805:90 , srv1501032106:90 , srv1516107263:90 , srv1560514153:90 , srv1643611248:90 , srv1700190074:90 , srv1890723544:90 , srv1899167711:90 , srv1904689054:90 , srv1950260687:90 , srv1989401090:90 , srv1990199606:90 , srv225834847:90 , srv278101039:90 , srv343098669:90 , srv366892392:90 , srv425822825:90 , srv484249641:90 , srv513966694:90 , srv536712643:90 , srv756570198:90 , srv826030085:90 , srv931672278:90 , srv939974148:90 , srv958003250:90 }
2023-05-12 20:55:28,132 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=208 (was 345), ProcessCount=171 (was 171), AvailableMemoryMB=9353 (was 9494)