2023-11-02 21:52:46,377 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-11-02 21:52:46,653 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-11-02 21:52:46,749 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-11-02 21:52:46,814 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-02 21:52:46,820 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-11-02 21:52:46,833 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=326, ProcessCount=175, AvailableMemoryMB=2280
2023-11-02 21:52:46,835 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-02 21:52:46,836 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-11-02 21:52:46,954 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv100562420:84 , srv1019128832:84 , srv1043810972:84 , srv1076020694:84 , srv1097427352:84 , srv1110322072:84 , srv1112316190:84 , srv1115971676:84 , srv1120519423:84 , srv1135775019:84 , srv1163956010:84 , srv1174067366:84 , srv1214070757:84 , srv1240871493:84 , srv1241117278:84 , srv1249829137:84 , srv1287567651:84 , srv1295377144:84 , srv1302028772:84 , srv1303576987:84 , srv1332091654:84 , srv1345317046:84 , srv1364751061:84 , srv1368925474:84 , srv1380674940:84 , srv1414070300:84 , srv1446101872:84 , srv1448042760:84 , srv1465229114:84 , srv1489018445:84 , srv151626578:84 , srv1531333154:84 , srv154006934:84 , srv1570352614:84 , srv1599241368:84 , srv1606753785:84 , srv1631143522:84 , srv1635505620:84 , srv1651381365:84 , srv16793252:84 , srv1682428346:84 , srv1696658813:84 , srv1750674232:84 , srv1774497713:84 , srv185196443:84 , srv1874344510:84 , srv1877753768:84 , srv1883706036:84 , srv1892701206:84 , srv1895595521:84 , srv1901588301:84 , srv1903822874:84 , srv190888645:84 , srv1920019967:84 , srv1925077499:84 , srv1967093385:84 , srv1967106041:84 , srv2047244186:84 , srv2087157125:84 , srv2104034293:84 , srv2122532408:84 , srv220225313:84 , srv258087883:84 , srv262898000:84 , srv285605128:84 , srv324589757:84 , srv36136650:84 , srv368943143:84 , srv382630992:84 , srv406640298:84 , srv423605869:84 , srv425330096:84 , srv448468812:84 , srv492007859:84 , srv542415169:84 , srv544605372:84 , srv545740955:84 , srv563185056:84 , srv591233949:84 , srv615941521:84 , srv660954736:84 , srv71403243:84 , srv727426210:84 , srv731875422:84 , srv760492868:84 , srv790079097:84 , srv808894847:84 , srv837975728:84 , srv841292347:84 , srv853426389:84 , srv877894943:84 , srv881807935:84 , srv910512899:84 , srv913276225:84 , srv917329338:84 , srv924080927:84 , srv924583407:84 , srv951648237:84 , srv957819184:84 , srv878894827:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-11-02 21:52:46,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-02 21:52:46,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2122532408=60, srv1110322072=5, srv1489018445=29, srv448468812=72, srv1570352614=33, srv71403243=81, srv881807935=92, srv151626578=30, srv1135775019=9, srv1214070757=12, srv2087157125=58, srv1895595521=49, srv837975728=87, srv1883706036=47, srv760492868=84, srv1874344510=45, srv1241117278=14, srv924583407=97, srv1414070300=25, srv2047244186=57, srv1112316190=6, srv406640298=69, srv100562420=0, srv1967106041=56, srv731875422=83, srv190888645=52, srv1877753768=46, srv285605128=64, srv544605372=75, srv877894943=90, srv1531333154=31, srv368943143=67, srv185196443=44, srv1019128832=1, srv1750674232=42, srv1599241368=34, srv808894847=86, srv1901588301=50, srv425330096=71, srv1635505620=37, srv1076020694=3, srv262898000=63, srv1295377144=17, srv924080927=96, srv591233949=78, srv220225313=61, srv790079097=85, srv1303576987=19, srv36136650=66, srv545740955=76, srv660954736=80, srv154006934=32, srv1967093385=55, srv917329338=95, srv1043810972=2, srv2104034293=59, srv1332091654=20, srv878894827=91, srv1903822874=51, srv1380674940=24, srv492007859=73, srv1097427352=4, srv542415169=74, srv1120519423=8, srv1631143522=36, srv1682428346=40, srv1696658813=41, srv1606753785=35, srv1249829137=15, srv615941521=79, srv913276225=94, srv853426389=89, srv1240871493=13, srv910512899=93, srv1364751061=22, srv1174067366=11, srv1446101872=26, srv16793252=39, srv324589757=65, srv1774497713=43, srv727426210=82, srv258087883=62, srv382630992=68, srv951648237=98, srv1345317046=21, srv423605869=70, srv1163956010=10, srv563185056=77, srv1115971676=7, srv1892701206=48, srv1448042760=27, srv1302028772=18, srv1920019967=53, srv1287567651=16, srv1465229114=28, srv841292347=88, srv1368925474=23, srv957819184=99, srv1651381365=38, srv1925077499=54} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-02 21:52:47,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-02 21:52:47,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-02 21:52:47,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-02 21:52:47,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-02 21:52:47,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-02 21:52:47,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-02 21:52:47,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-02 21:52:47,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-02 21:52:47,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-02 21:52:47,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-02 21:52:47,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-02 21:52:47,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-02 21:52:47,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-02 21:52:47,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-02 21:52:47,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-02 21:52:47,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-02 21:52:47,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-02 21:52:47,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-02 21:52:47,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-02 21:52:47,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-02 21:52:47,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-02 21:52:47,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-02 21:52:47,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-02 21:52:47,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-02 21:52:47,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-02 21:52:47,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-02 21:52:47,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-02 21:52:47,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-02 21:52:47,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-02 21:52:47,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-02 21:52:47,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-02 21:52:47,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-02 21:52:47,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-02 21:52:47,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-02 21:52:47,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-02 21:52:47,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-02 21:52:47,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-02 21:52:47,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-02 21:52:47,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-02 21:52:47,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-02 21:52:47,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-02 21:52:47,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-02 21:52:47,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-02 21:52:47,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-02 21:52:47,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-02 21:52:47,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-02 21:52:47,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-02 21:52:47,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-02 21:52:47,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-02 21:52:47,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-02 21:52:47,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-02 21:52:47,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-02 21:52:47,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-02 21:52:47,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-02 21:52:47,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-02 21:52:47,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-02 21:52:47,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-02 21:52:47,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-02 21:52:47,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-02 21:52:47,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-02 21:52:47,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-02 21:52:47,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-02 21:52:47,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-02 21:52:47,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-02 21:52:47,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-02 21:52:47,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-02 21:52:47,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-02 21:52:47,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-02 21:52:47,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-02 21:52:47,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-02 21:52:47,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-02 21:52:47,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-02 21:52:47,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-02 21:52:47,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-02 21:52:47,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-02 21:52:47,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-02 21:52:47,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-02 21:52:47,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-02 21:52:47,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-02 21:52:47,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-02 21:52:47,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-02 21:52:47,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-02 21:52:47,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-02 21:52:47,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-02 21:52:47,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-02 21:52:47,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-02 21:52:47,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-02 21:52:47,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-02 21:52:47,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-02 21:52:47,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-02 21:52:47,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-02 21:52:47,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-02 21:52:47,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-02 21:52:47,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-02 21:52:47,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-02 21:52:47,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-02 21:52:47,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-02 21:52:47,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-02 21:52:47,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-02 21:52:47,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-02 21:52:47,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-02 21:52:47,472 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-02 21:52:47,472 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-02 21:52:47,473 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-02 21:52:47,964 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 873 ms to try 5000 different iterations.  Found a solution that moves 4698 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17719379980156752. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.038715824628862736, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.058910802097883005, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.522, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.038715824628862736, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17208333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.242, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-02 21:52:48,007 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv2122532408:80 , srv1110322072:82 , srv1302028772:82 , srv1414070300:82 , srv1883706036:82 , srv1635505620:83 , srv1892701206:83 , srv1920019967:83 , srv258087883:83 , srv760492868:83 , srv910512899:83 , srv1019128832:84 , srv1368925474:84 , srv1465229114:84 , srv16793252:84 , srv1682428346:84 , srv190888645:84 , srv2047244186:84 , srv382630992:84 , srv837975728:84 , srv1112316190:85 , srv1174067366:85 , srv1214070757:85 , srv1240871493:85 , srv1287567651:85 , srv1303576987:85 , srv1332091654:85 , srv1345317046:85 , srv151626578:85 , srv1570352614:85 , srv1651381365:85 , srv1901588301:85 , srv808894847:85 , srv924080927:85 , srv1043810972:86 , srv1076020694:86 , srv1120519423:86 , srv1295377144:86 , srv1380674940:86 , srv1489018445:86 , srv1877753768:86 , srv220225313:86 , srv36136650:86 , srv368943143:86 , srv492007859:86 , srv71403243:86 , srv790079097:86 , srv853426389:86 , srv877894943:86 , srv881807935:86 , srv951648237:86 , srv100562420:87 , srv1097427352:87 , srv1115971676:87 , srv1135775019:87 , srv1241117278:87 , srv1249829137:87 , srv154006934:87 , srv185196443:87 , srv1895595521:87 , srv1925077499:87 , srv1967093385:87 , srv1967106041:87 , srv262898000:87 , srv423605869:87 , srv917329338:87 , srv924583407:87 , srv1448042760:88 , srv1531333154:88 , srv1696658813:88 , srv1750674232:88 , srv1874344510:88 , srv1903822874:88 , srv2087157125:88 , srv324589757:88 , srv406640298:88 , srv544605372:88 , srv563185056:88 , srv660954736:88 , srv727426210:88 , srv841292347:88 , srv957819184:88 , srv2104034293:89 , srv425330096:89 , srv448468812:89 , srv542415169:89 , srv545740955:89 , srv913276225:89 , srv1446101872:90 , srv1599241368:90 , srv1631143522:90 , srv285605128:90 , srv615941521:90 , srv1163956010:91 , srv1364751061:91 , srv1606753785:92 , srv1774497713:93 , srv591233949:94 , srv731875422:95 , srv878894827:434 }
2023-11-02 21:52:48,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-02 21:52:48,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2122532408=60, srv1110322072=5, srv1489018445=29, srv448468812=72, srv1570352614=33, srv71403243=81, srv881807935=92, srv151626578=30, srv1135775019=9, srv1214070757=12, srv2087157125=58, srv1895595521=49, srv837975728=87, srv1883706036=47, srv760492868=84, srv1874344510=45, srv1241117278=14, srv924583407=97, srv1414070300=25, srv2047244186=57, srv1112316190=6, srv406640298=69, srv100562420=0, srv1967106041=56, srv731875422=83, srv190888645=52, srv1877753768=46, srv285605128=64, srv544605372=75, srv877894943=90, srv1531333154=31, srv368943143=67, srv185196443=44, srv1019128832=1, srv1750674232=42, srv1599241368=34, srv808894847=86, srv1901588301=50, srv425330096=71, srv1635505620=37, srv1076020694=3, srv262898000=63, srv1295377144=17, srv924080927=96, srv591233949=78, srv220225313=61, srv790079097=85, srv1303576987=19, srv36136650=66, srv545740955=76, srv660954736=80, srv154006934=32, srv1967093385=55, srv917329338=95, srv1043810972=2, srv2104034293=59, srv1332091654=20, srv878894827=91, srv1903822874=51, srv1380674940=24, srv492007859=73, srv1097427352=4, srv542415169=74, srv1120519423=8, srv1631143522=36, srv1682428346=40, srv1696658813=41, srv1606753785=35, srv1249829137=15, srv615941521=79, srv913276225=94, srv853426389=89, srv1240871493=13, srv910512899=93, srv1364751061=22, srv1174067366=11, srv1446101872=26, srv16793252=39, srv324589757=65, srv1774497713=43, srv727426210=82, srv258087883=62, srv382630992=68, srv951648237=98, srv1345317046=21, srv423605869=70, srv1163956010=10, srv563185056=77, srv1115971676=7, srv1892701206=48, srv1448042760=27, srv1302028772=18, srv1920019967=53, srv1287567651=16, srv1465229114=28, srv841292347=88, srv1368925474=23, srv957819184=99, srv1651381365=38, srv1925077499=54} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-02 21:52:48,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-02 21:52:48,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-02 21:52:48,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-02 21:52:48,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-02 21:52:48,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-02 21:52:48,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-02 21:52:48,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-02 21:52:48,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-02 21:52:48,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-02 21:52:48,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-02 21:52:48,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-02 21:52:48,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-02 21:52:48,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-02 21:52:48,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-02 21:52:48,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-02 21:52:48,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-02 21:52:48,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-02 21:52:48,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-02 21:52:48,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-02 21:52:48,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-02 21:52:48,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-02 21:52:48,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-02 21:52:48,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-02 21:52:48,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-02 21:52:48,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-02 21:52:48,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-02 21:52:48,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-02 21:52:48,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-02 21:52:48,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-02 21:52:48,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-02 21:52:48,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-02 21:52:48,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-02 21:52:48,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-02 21:52:48,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-02 21:52:48,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-02 21:52:48,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-02 21:52:48,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-02 21:52:48,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-02 21:52:48,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-02 21:52:48,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-02 21:52:48,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-02 21:52:48,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-02 21:52:48,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-02 21:52:48,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-02 21:52:48,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-02 21:52:48,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-02 21:52:48,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-02 21:52:48,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-02 21:52:48,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-02 21:52:48,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-02 21:52:48,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-02 21:52:48,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-02 21:52:48,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-02 21:52:48,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-02 21:52:48,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-02 21:52:48,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-02 21:52:48,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-02 21:52:48,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-02 21:52:48,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-02 21:52:48,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-02 21:52:48,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-02 21:52:48,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-02 21:52:48,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-02 21:52:48,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-02 21:52:48,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-02 21:52:48,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-02 21:52:48,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-02 21:52:48,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-02 21:52:48,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-02 21:52:48,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-02 21:52:48,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-02 21:52:48,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-02 21:52:48,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-02 21:52:48,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-02 21:52:48,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-02 21:52:48,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-02 21:52:48,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-02 21:52:48,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-02 21:52:48,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-02 21:52:48,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-02 21:52:48,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-02 21:52:48,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-02 21:52:48,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-02 21:52:48,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-02 21:52:48,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-02 21:52:48,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-02 21:52:48,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-02 21:52:48,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-02 21:52:48,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-02 21:52:48,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-02 21:52:48,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-02 21:52:48,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-02 21:52:48,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-02 21:52:48,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-02 21:52:48,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-02 21:52:48,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-02 21:52:48,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-02 21:52:48,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-02 21:52:48,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-02 21:52:48,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-02 21:52:48,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-02 21:52:48,228 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.038715824628862736, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.058910802097883005, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.038715824628862736, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17208333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.242, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-02 21:52:48,229 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-02 21:52:48,229 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.1771609037024651, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.038715824628862736, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.058910802097883005, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.038715824628862736, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17208333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.242, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-02 21:52:48,462 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 372 ms to try 5000 different iterations.  Found a solution that moves 2687 regions; Going from a computed imbalance of 0.1771609037024651 to a new imbalance of 0.013931627931423829. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020250950844368027); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0357852429629442, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2985555555555556, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020250950844368027); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01075); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04416666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-02 21:52:48,473 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1774497713:77 , srv591233949:78 , srv731875422:79 , srv1163956010:80 , srv1446101872:81 , srv1631143522:81 , srv615941521:81 , srv1364751061:82 , srv1606753785:82 , srv1903822874:82 , srv2104034293:82 , srv285605128:82 , srv406640298:82 , srv1448042760:83 , srv1599241368:83 , srv262898000:83 , srv542415169:83 , srv660954736:83 , srv1043810972:84 , srv1115971676:84 , srv1135775019:84 , srv1241117278:84 , srv1249829137:84 , srv1295377144:84 , srv154006934:84 , srv1696658813:84 , srv448468812:84 , srv563185056:84 , srv853426389:84 , srv913276225:84 , srv924583407:84 , srv957819184:84 , srv1345317046:85 , srv1531333154:85 , srv1750674232:85 , srv185196443:85 , srv1874344510:85 , srv2087157125:85 , srv324589757:85 , srv423605869:85 , srv425330096:85 , srv544605372:85 , srv545740955:85 , srv71403243:85 , srv727426210:85 , srv917329338:85 , srv951648237:85 , srv100562420:86 , srv1076020694:86 , srv1174067366:86 , srv1240871493:86 , srv1332091654:86 , srv1877753768:86 , srv1895595521:86 , srv1967093385:86 , srv220225313:86 , srv368943143:86 , srv492007859:86 , srv1120519423:87 , srv1303576987:87 , srv1380674940:87 , srv1489018445:87 , srv1651381365:87 , srv190888645:87 , srv1925077499:87 , srv1967106041:87 , srv2047244186:87 , srv36136650:87 , srv760492868:87 , srv808894847:87 , srv837975728:87 , srv841292347:87 , srv877894943:87 , srv1097427352:88 , srv1112316190:88 , srv1214070757:88 , srv1287567651:88 , srv1368925474:88 , srv1465229114:88 , srv151626578:88 , srv1570352614:88 , srv1635505620:88 , srv382630992:88 , srv790079097:88 , srv881807935:88 , srv910512899:88 , srv1414070300:89 , srv16793252:89 , srv1682428346:89 , srv1901588301:89 , srv1920019967:89 , srv258087883:89 , srv1883706036:90 , srv1892701206:90 , srv924080927:90 , srv1019128832:91 , srv1110322072:91 , srv1302028772:91 , srv2122532408:93 , srv878894827:520 }
2023-11-02 21:52:48,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-02 21:52:48,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2122532408=60, srv1110322072=5, srv1489018445=29, srv448468812=72, srv1570352614=33, srv71403243=81, srv881807935=92, srv151626578=30, srv1135775019=9, srv1214070757=12, srv2087157125=58, srv1895595521=49, srv837975728=87, srv1883706036=47, srv760492868=84, srv1874344510=45, srv1241117278=14, srv924583407=97, srv1414070300=25, srv2047244186=57, srv1112316190=6, srv406640298=69, srv100562420=0, srv1967106041=56, srv731875422=83, srv190888645=52, srv1877753768=46, srv285605128=64, srv544605372=75, srv877894943=90, srv1531333154=31, srv368943143=67, srv185196443=44, srv1019128832=1, srv1750674232=42, srv1599241368=34, srv808894847=86, srv1901588301=50, srv425330096=71, srv1635505620=37, srv1076020694=3, srv262898000=63, srv1295377144=17, srv924080927=96, srv591233949=78, srv220225313=61, srv790079097=85, srv1303576987=19, srv36136650=66, srv545740955=76, srv660954736=80, srv154006934=32, srv1967093385=55, srv917329338=95, srv1043810972=2, srv2104034293=59, srv1332091654=20, srv878894827=91, srv1903822874=51, srv1380674940=24, srv492007859=73, srv1097427352=4, srv542415169=74, srv1120519423=8, srv1631143522=36, srv1682428346=40, srv1696658813=41, srv1606753785=35, srv1249829137=15, srv615941521=79, srv913276225=94, srv853426389=89, srv1240871493=13, srv910512899=93, srv1364751061=22, srv1174067366=11, srv1446101872=26, srv16793252=39, srv324589757=65, srv1774497713=43, srv727426210=82, srv258087883=62, srv382630992=68, srv951648237=98, srv1345317046=21, srv423605869=70, srv1163956010=10, srv563185056=77, srv1115971676=7, srv1892701206=48, srv1448042760=27, srv1302028772=18, srv1920019967=53, srv1287567651=16, srv1465229114=28, srv841292347=88, srv1368925474=23, srv957819184=99, srv1651381365=38, srv1925077499=54} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-02 21:52:48,504 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-02 21:52:48,505 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-02 21:52:48,505 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-02 21:52:48,505 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-02 21:52:48,505 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-02 21:52:48,505 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-02 21:52:48,505 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-02 21:52:48,505 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-02 21:52:48,505 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-02 21:52:48,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-02 21:52:48,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-02 21:52:48,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-02 21:52:48,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-02 21:52:48,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-02 21:52:48,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-02 21:52:48,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-02 21:52:48,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-02 21:52:48,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-02 21:52:48,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-02 21:52:48,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-02 21:52:48,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-02 21:52:48,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-02 21:52:48,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-02 21:52:48,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-02 21:52:48,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-02 21:52:48,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-02 21:52:48,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-02 21:52:48,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-02 21:52:48,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-02 21:52:48,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-02 21:52:48,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-02 21:52:48,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-02 21:52:48,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-02 21:52:48,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-02 21:52:48,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-02 21:52:48,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-02 21:52:48,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-02 21:52:48,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-02 21:52:48,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-02 21:52:48,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-02 21:52:48,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-02 21:52:48,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-02 21:52:48,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-02 21:52:48,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-02 21:52:48,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-02 21:52:48,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-02 21:52:48,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-02 21:52:48,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-02 21:52:48,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-02 21:52:48,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-02 21:52:48,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-02 21:52:48,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-02 21:52:48,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-02 21:52:48,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-02 21:52:48,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-02 21:52:48,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-02 21:52:48,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-02 21:52:48,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-02 21:52:48,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-02 21:52:48,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-02 21:52:48,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-02 21:52:48,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-02 21:52:48,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-02 21:52:48,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-02 21:52:48,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-02 21:52:48,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-02 21:52:48,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-02 21:52:48,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-02 21:52:48,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-02 21:52:48,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-02 21:52:48,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-02 21:52:48,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-02 21:52:48,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-02 21:52:48,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-02 21:52:48,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-02 21:52:48,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-02 21:52:48,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-02 21:52:48,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-02 21:52:48,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-02 21:52:48,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-02 21:52:48,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-02 21:52:48,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-02 21:52:48,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-02 21:52:48,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-02 21:52:48,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-02 21:52:48,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-02 21:52:48,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-02 21:52:48,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-02 21:52:48,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-02 21:52:48,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-02 21:52:48,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-02 21:52:48,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-02 21:52:48,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-02 21:52:48,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-02 21:52:48,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-02 21:52:48,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-02 21:52:48,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-02 21:52:48,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-02 21:52:48,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-02 21:52:48,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-02 21:52:48,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-02 21:52:48,612 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020250950844368027); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0357852429629442, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020250950844368027); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01075); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04416666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-02 21:52:48,613 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-02 21:52:48,613 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.013912813155287555, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020250950844368027); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0357852429629442, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020250950844368027); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01075); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04416666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-02 21:52:48,797 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 268 ms to try 5000 different iterations.  Found a solution that moves 1028 regions; Going from a computed imbalance of 0.013912813155287555 to a new imbalance of 3.4623485786508014E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005997047949844344); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008901506196714459); MoveCostFunction : (multiplier=7.0, imbalance=0.11422222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005997047949844344); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-02 21:52:48,799 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1019128832:82 , srv924080927:82 , srv1097427352:83 , srv1901588301:83 , srv1967106041:83 , srv425330096:83 , srv545740955:83 , srv731875422:83 , srv790079097:83 , srv841292347:83 , srv1110322072:84 , srv1112316190:84 , srv1120519423:84 , srv1214070757:84 , srv1287567651:84 , srv1302028772:84 , srv1380674940:84 , srv1531333154:84 , srv1570352614:84 , srv1606753785:84 , srv16793252:84 , srv1696658813:84 , srv1892701206:84 , srv1895595521:84 , srv1925077499:84 , srv1967093385:84 , srv2122532408:84 , srv324589757:84 , srv36136650:84 , srv544605372:84 , srv881807935:84 , srv100562420:85 , srv1303576987:85 , srv1364751061:85 , srv1465229114:85 , srv1489018445:85 , srv151626578:85 , srv1599241368:85 , srv1682428346:85 , srv1750674232:85 , srv1874344510:85 , srv1877753768:85 , srv1883706036:85 , srv1920019967:85 , srv220225313:85 , srv258087883:85 , srv285605128:85 , srv368943143:85 , srv382630992:85 , srv448468812:85 , srv542415169:85 , srv727426210:85 , srv913276225:85 , srv917329338:85 , srv1076020694:86 , srv1135775019:86 , srv1163956010:86 , srv1174067366:86 , srv1249829137:86 , srv1448042760:86 , srv1635505620:86 , srv1651381365:86 , srv190888645:86 , srv2047244186:86 , srv2104034293:86 , srv262898000:86 , srv423605869:86 , srv492007859:86 , srv563185056:86 , srv591233949:86 , srv615941521:86 , srv808894847:86 , srv877894943:86 , srv910512899:86 , srv924583407:86 , srv951648237:86 , srv957819184:86 , srv1043810972:87 , srv1240871493:87 , srv1295377144:87 , srv1332091654:87 , srv1345317046:87 , srv1368925474:87 , srv1414070300:87 , srv1446101872:87 , srv154006934:87 , srv1631143522:87 , srv1774497713:87 , srv185196443:87 , srv2087157125:87 , srv406640298:87 , srv660954736:87 , srv760492868:87 , srv837975728:87 , srv853426389:87 , srv1115971676:88 , srv1241117278:88 , srv1903822874:88 , srv71403243:88 , srv878894827:557 }
2023-11-02 21:52:48,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-02 21:52:48,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2122532408=60, srv1110322072=5, srv1489018445=29, srv448468812=72, srv1570352614=33, srv71403243=81, srv881807935=92, srv151626578=30, srv1135775019=9, srv1214070757=12, srv2087157125=58, srv1895595521=49, srv837975728=87, srv1883706036=47, srv760492868=84, srv1874344510=45, srv1241117278=14, srv924583407=97, srv1414070300=25, srv2047244186=57, srv1112316190=6, srv406640298=69, srv100562420=0, srv1967106041=56, srv731875422=83, srv190888645=52, srv1877753768=46, srv285605128=64, srv544605372=75, srv877894943=90, srv1531333154=31, srv368943143=67, srv185196443=44, srv1019128832=1, srv1750674232=42, srv1599241368=34, srv808894847=86, srv1901588301=50, srv425330096=71, srv1635505620=37, srv1076020694=3, srv262898000=63, srv1295377144=17, srv924080927=96, srv591233949=78, srv220225313=61, srv790079097=85, srv1303576987=19, srv36136650=66, srv545740955=76, srv660954736=80, srv154006934=32, srv1967093385=55, srv917329338=95, srv1043810972=2, srv2104034293=59, srv1332091654=20, srv878894827=91, srv1903822874=51, srv1380674940=24, srv492007859=73, srv1097427352=4, srv542415169=74, srv1120519423=8, srv1631143522=36, srv1682428346=40, srv1696658813=41, srv1606753785=35, srv1249829137=15, srv615941521=79, srv913276225=94, srv853426389=89, srv1240871493=13, srv910512899=93, srv1364751061=22, srv1174067366=11, srv1446101872=26, srv16793252=39, srv324589757=65, srv1774497713=43, srv727426210=82, srv258087883=62, srv382630992=68, srv951648237=98, srv1345317046=21, srv423605869=70, srv1163956010=10, srv563185056=77, srv1115971676=7, srv1892701206=48, srv1448042760=27, srv1302028772=18, srv1920019967=53, srv1287567651=16, srv1465229114=28, srv841292347=88, srv1368925474=23, srv957819184=99, srv1651381365=38, srv1925077499=54} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-02 21:52:48,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-02 21:52:48,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-02 21:52:48,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-02 21:52:48,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-02 21:52:48,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-02 21:52:48,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-02 21:52:48,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-02 21:52:48,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-02 21:52:48,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-02 21:52:48,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-02 21:52:48,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-02 21:52:48,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-02 21:52:48,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-02 21:52:48,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-02 21:52:48,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-02 21:52:48,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-02 21:52:48,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-02 21:52:48,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-02 21:52:48,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-02 21:52:48,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-02 21:52:48,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-02 21:52:48,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-02 21:52:48,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-02 21:52:48,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-02 21:52:48,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-02 21:52:48,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-02 21:52:48,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-02 21:52:48,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-02 21:52:48,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-02 21:52:48,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-02 21:52:48,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-02 21:52:48,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-02 21:52:48,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-02 21:52:48,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-02 21:52:48,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-02 21:52:48,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-02 21:52:48,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-02 21:52:48,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-02 21:52:48,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-02 21:52:48,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-02 21:52:48,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-02 21:52:48,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-02 21:52:48,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-02 21:52:48,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-02 21:52:48,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-02 21:52:48,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-02 21:52:48,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-02 21:52:48,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-02 21:52:48,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-02 21:52:48,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-02 21:52:48,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-02 21:52:48,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-02 21:52:48,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-02 21:52:48,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-02 21:52:48,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-02 21:52:48,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-02 21:52:48,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-02 21:52:48,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-02 21:52:48,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-02 21:52:48,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-02 21:52:48,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-02 21:52:48,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-02 21:52:48,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-02 21:52:48,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-02 21:52:48,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-02 21:52:48,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-02 21:52:48,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-02 21:52:48,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-02 21:52:48,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-02 21:52:48,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-02 21:52:48,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-02 21:52:48,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-02 21:52:48,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-02 21:52:48,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-02 21:52:48,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-02 21:52:48,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-02 21:52:48,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-02 21:52:48,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-02 21:52:48,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-02 21:52:48,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-02 21:52:48,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-02 21:52:48,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-02 21:52:48,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-02 21:52:48,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-02 21:52:48,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-02 21:52:48,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-02 21:52:48,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-02 21:52:48,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-02 21:52:48,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-02 21:52:48,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-02 21:52:48,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-02 21:52:48,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-02 21:52:48,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-02 21:52:48,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-02 21:52:48,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-02 21:52:48,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-02 21:52:48,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-02 21:52:48,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-02 21:52:48,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-02 21:52:48,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-02 21:52:48,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-02 21:52:48,925 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005997047949844344); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008901506196714459); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005997047949844344); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-02 21:52:48,925 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-02 21:52:48,925 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=3.3903664801465607E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005997047949844344); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008901506196714459); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005997047949844344); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-02 21:52:49,250 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 414 ms to try 5000 different iterations.  Found a solution that moves 217 regions; Going from a computed imbalance of 3.3903664801465607E-4 to a new imbalance of 1.8982397451953428E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.5313429231318235E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.02411111111111111); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.5313429231318235E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-02 21:52:49,251 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv2087157125:82 , srv1115971676:83 , srv1241117278:83 , srv1368925474:83 , srv185196443:83 , srv71403243:83 , srv877894943:83 , srv100562420:84 , srv1076020694:84 , srv1097427352:84 , srv1240871493:84 , srv1332091654:84 , srv1364751061:84 , srv1414070300:84 , srv1446101872:84 , srv1489018445:84 , srv151626578:84 , srv154006934:84 , srv1599241368:84 , srv1606753785:84 , srv1631143522:84 , srv1651381365:84 , srv1682428346:84 , srv1750674232:84 , srv1774497713:84 , srv1874344510:84 , srv1903822874:84 , srv190888645:84 , srv1925077499:84 , srv2047244186:84 , srv423605869:84 , srv448468812:84 , srv492007859:84 , srv563185056:84 , srv591233949:84 , srv615941521:84 , srv660954736:84 , srv727426210:84 , srv808894847:84 , srv837975728:84 , srv881807935:84 , srv913276225:84 , srv951648237:84 , srv957819184:84 , srv1019128832:85 , srv1043810972:85 , srv1110322072:85 , srv1112316190:85 , srv1120519423:85 , srv1135775019:85 , srv1163956010:85 , srv1174067366:85 , srv1214070757:85 , srv1249829137:85 , srv1287567651:85 , srv1295377144:85 , srv1302028772:85 , srv1303576987:85 , srv1345317046:85 , srv1380674940:85 , srv1448042760:85 , srv1465229114:85 , srv1531333154:85 , srv1635505620:85 , srv16793252:85 , srv1883706036:85 , srv1895595521:85 , srv1901588301:85 , srv1920019967:85 , srv1967093385:85 , srv2104034293:85 , srv2122532408:85 , srv220225313:85 , srv258087883:85 , srv285605128:85 , srv324589757:85 , srv36136650:85 , srv368943143:85 , srv382630992:85 , srv406640298:85 , srv425330096:85 , srv542415169:85 , srv544605372:85 , srv545740955:85 , srv731875422:85 , srv760492868:85 , srv790079097:85 , srv841292347:85 , srv853426389:85 , srv910512899:85 , srv917329338:85 , srv924080927:85 , srv924583407:85 , srv1570352614:86 , srv1696658813:86 , srv1877753768:86 , srv1892701206:86 , srv1967106041:86 , srv262898000:86 , srv878894827:631 }
2023-11-02 21:52:49,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-02 21:52:49,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2122532408=60, srv1110322072=5, srv1489018445=29, srv448468812=72, srv1570352614=33, srv71403243=81, srv881807935=92, srv151626578=30, srv1135775019=9, srv1214070757=12, srv2087157125=58, srv1895595521=49, srv837975728=87, srv1883706036=47, srv760492868=84, srv1874344510=45, srv1241117278=14, srv924583407=97, srv1414070300=25, srv2047244186=57, srv1112316190=6, srv406640298=69, srv100562420=0, srv1967106041=56, srv731875422=83, srv190888645=52, srv1877753768=46, srv285605128=64, srv544605372=75, srv877894943=90, srv1531333154=31, srv368943143=67, srv185196443=44, srv1019128832=1, srv1750674232=42, srv1599241368=34, srv808894847=86, srv1901588301=50, srv425330096=71, srv1635505620=37, srv1076020694=3, srv262898000=63, srv1295377144=17, srv924080927=96, srv591233949=78, srv220225313=61, srv790079097=85, srv1303576987=19, srv36136650=66, srv545740955=76, srv660954736=80, srv154006934=32, srv1967093385=55, srv917329338=95, srv1043810972=2, srv2104034293=59, srv1332091654=20, srv878894827=91, srv1903822874=51, srv1380674940=24, srv492007859=73, srv1097427352=4, srv542415169=74, srv1120519423=8, srv1631143522=36, srv1682428346=40, srv1696658813=41, srv1606753785=35, srv1249829137=15, srv615941521=79, srv913276225=94, srv853426389=89, srv1240871493=13, srv910512899=93, srv1364751061=22, srv1174067366=11, srv1446101872=26, srv16793252=39, srv324589757=65, srv1774497713=43, srv727426210=82, srv258087883=62, srv382630992=68, srv951648237=98, srv1345317046=21, srv423605869=70, srv1163956010=10, srv563185056=77, srv1115971676=7, srv1892701206=48, srv1448042760=27, srv1302028772=18, srv1920019967=53, srv1287567651=16, srv1465229114=28, srv841292347=88, srv1368925474=23, srv957819184=99, srv1651381365=38, srv1925077499=54} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-02 21:52:49,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-02 21:52:49,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-02 21:52:49,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-02 21:52:49,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-02 21:52:49,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-02 21:52:49,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-02 21:52:49,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-02 21:52:49,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-02 21:52:49,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-02 21:52:49,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-02 21:52:49,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-02 21:52:49,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-02 21:52:49,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-02 21:52:49,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-02 21:52:49,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-02 21:52:49,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-02 21:52:49,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-02 21:52:49,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-02 21:52:49,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-02 21:52:49,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-02 21:52:49,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-02 21:52:49,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-02 21:52:49,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-02 21:52:49,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-02 21:52:49,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-02 21:52:49,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-02 21:52:49,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-02 21:52:49,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-02 21:52:49,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-02 21:52:49,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-02 21:52:49,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-02 21:52:49,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-02 21:52:49,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-02 21:52:49,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-02 21:52:49,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-02 21:52:49,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-02 21:52:49,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-02 21:52:49,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-02 21:52:49,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-02 21:52:49,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-02 21:52:49,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-02 21:52:49,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-02 21:52:49,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-02 21:52:49,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-02 21:52:49,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-02 21:52:49,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-02 21:52:49,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-02 21:52:49,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-02 21:52:49,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-02 21:52:49,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-02 21:52:49,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-02 21:52:49,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-02 21:52:49,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-02 21:52:49,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-02 21:52:49,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-02 21:52:49,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-02 21:52:49,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-02 21:52:49,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-02 21:52:49,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-02 21:52:49,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-02 21:52:49,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-02 21:52:49,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-02 21:52:49,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-02 21:52:49,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-02 21:52:49,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-02 21:52:49,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-02 21:52:49,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-02 21:52:49,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-02 21:52:49,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-02 21:52:49,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-02 21:52:49,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-02 21:52:49,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-02 21:52:49,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-02 21:52:49,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-02 21:52:49,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-02 21:52:49,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-02 21:52:49,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-02 21:52:49,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-02 21:52:49,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-02 21:52:49,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-02 21:52:49,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-02 21:52:49,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-02 21:52:49,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-02 21:52:49,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-02 21:52:49,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-02 21:52:49,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-02 21:52:49,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-02 21:52:49,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-02 21:52:49,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-02 21:52:49,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-02 21:52:49,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-02 21:52:49,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-02 21:52:49,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-02 21:52:49,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-02 21:52:49,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-02 21:52:49,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-02 21:52:49,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-02 21:52:49,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-02 21:52:49,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-02 21:52:49,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-02 21:52:49,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-02 21:52:49,402 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.7462930975745232E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.5313429231318235E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.5313429231318235E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-02 21:52:49,402 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv2087157125:82 , srv1115971676:83 , srv1241117278:83 , srv1368925474:83 , srv185196443:83 , srv71403243:83 , srv877894943:83 , srv100562420:84 , srv1076020694:84 , srv1097427352:84 , srv1240871493:84 , srv1332091654:84 , srv1364751061:84 , srv1414070300:84 , srv1446101872:84 , srv1489018445:84 , srv151626578:84 , srv154006934:84 , srv1599241368:84 , srv1606753785:84 , srv1631143522:84 , srv1651381365:84 , srv1682428346:84 , srv1750674232:84 , srv1774497713:84 , srv1874344510:84 , srv1903822874:84 , srv190888645:84 , srv1925077499:84 , srv2047244186:84 , srv423605869:84 , srv448468812:84 , srv492007859:84 , srv563185056:84 , srv591233949:84 , srv615941521:84 , srv660954736:84 , srv727426210:84 , srv808894847:84 , srv837975728:84 , srv881807935:84 , srv913276225:84 , srv951648237:84 , srv957819184:84 , srv1019128832:85 , srv1043810972:85 , srv1110322072:85 , srv1112316190:85 , srv1120519423:85 , srv1135775019:85 , srv1163956010:85 , srv1174067366:85 , srv1214070757:85 , srv1249829137:85 , srv1287567651:85 , srv1295377144:85 , srv1302028772:85 , srv1303576987:85 , srv1345317046:85 , srv1380674940:85 , srv1448042760:85 , srv1465229114:85 , srv1531333154:85 , srv1635505620:85 , srv16793252:85 , srv1883706036:85 , srv1895595521:85 , srv1901588301:85 , srv1920019967:85 , srv1967093385:85 , srv2104034293:85 , srv2122532408:85 , srv220225313:85 , srv258087883:85 , srv285605128:85 , srv324589757:85 , srv36136650:85 , srv368943143:85 , srv382630992:85 , srv406640298:85 , srv425330096:85 , srv542415169:85 , srv544605372:85 , srv545740955:85 , srv731875422:85 , srv760492868:85 , srv790079097:85 , srv841292347:85 , srv853426389:85 , srv910512899:85 , srv917329338:85 , srv924080927:85 , srv924583407:85 , srv1570352614:86 , srv1696658813:86 , srv1877753768:86 , srv1892701206:86 , srv1967106041:86 , srv262898000:86 , srv878894827:631 }
2023-11-02 21:52:49,451 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=364 (was 326) - SystemLoadAverage LEAK? -, ProcessCount=175 (was 175), AvailableMemoryMB=1605 (was 2280)
2023-11-02 21:52:49,460 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=364, ProcessCount=175, AvailableMemoryMB=1604
2023-11-02 21:52:49,461 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-02 21:52:49,462 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-11-02 21:52:49,495 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1332688651:84 , srv1412599966:84 , srv1447522566:84 , srv1478956867:84 , srv1494514965:84 , srv1550799395:84 , srv1682469068:84 , srv1739629051:84 , srv1830397736:84 , srv197079212:84 , srv1975069627:84 , srv2077673903:84 , srv2130867416:84 , srv2147088810:84 , srv288570378:84 , srv418378136:84 , srv436594080:84 , srv45770472:84 , srv47489465:84 , srv505577208:84 , srv59733377:84 , srv681991050:84 , srv683282677:84 , srv68810782:84 , srv758966085:84 , srv771491520:84 , srv775437376:84 , srv926957575:84 , srv932428476:84 , srv2060591026:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-11-02 21:52:49,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-02 21:52:49,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv418378136=16, srv47489465=19, srv68810782=24, srv197079212=9, srv1478956867=3, srv683282677=23, srv1830397736=8, srv1975069627=10, srv2060591026=11, srv758966085=25, srv2077673903=12, srv1682469068=6, srv775437376=27, srv436594080=17, srv1412599966=1, srv288570378=15, srv59733377=21, srv45770472=18, srv1447522566=2, srv1332688651=0, srv1494514965=4, srv2130867416=13, srv926957575=28, srv681991050=22, srv1550799395=5, srv1739629051=7, srv505577208=20, srv771491520=26, srv932428476=29, srv2147088810=14} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-02 21:52:49,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-02 21:52:49,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-02 21:52:49,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-02 21:52:49,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-02 21:52:49,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-02 21:52:49,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-02 21:52:49,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-02 21:52:49,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-02 21:52:49,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-02 21:52:49,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-02 21:52:49,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-02 21:52:49,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-02 21:52:49,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-02 21:52:49,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-02 21:52:49,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-02 21:52:49,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-02 21:52:49,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-02 21:52:49,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-02 21:52:49,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-02 21:52:49,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-02 21:52:49,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-02 21:52:49,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-02 21:52:49,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-02 21:52:49,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-02 21:52:49,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-02 21:52:49,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-02 21:52:49,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-02 21:52:49,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-02 21:52:49,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-02 21:52:49,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-02 21:52:49,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-11-02 21:52:49,543 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1061682768460264, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-02 21:52:49,544 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912527576037908, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1061682768460264, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-11-02 21:54:49,532 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 9582555 different iterations.  Found a solution that moves 1831 regions; Going from a computed imbalance of 0.9912527576037908 to a new imbalance of 4.2736453424534666E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6781481481481482, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-02 21:54:49,537 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1332688651:90 , srv1412599966:90 , srv1447522566:90 , srv1478956867:90 , srv1494514965:90 , srv1550799395:90 , srv1682469068:90 , srv1739629051:90 , srv1830397736:90 , srv197079212:90 , srv1975069627:90 , srv2060591026:90 , srv2077673903:90 , srv2130867416:90 , srv2147088810:90 , srv288570378:90 , srv418378136:90 , srv436594080:90 , srv45770472:90 , srv47489465:90 , srv505577208:90 , srv59733377:90 , srv681991050:90 , srv683282677:90 , srv68810782:90 , srv758966085:90 , srv771491520:90 , srv775437376:90 , srv926957575:90 , srv932428476:90 }
2023-11-02 21:54:49,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-02 21:54:49,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv418378136=16, srv47489465=19, srv68810782=24, srv197079212=9, srv1478956867=3, srv683282677=23, srv1830397736=8, srv1975069627=10, srv2060591026=11, srv758966085=25, srv2077673903=12, srv1682469068=6, srv775437376=27, srv436594080=17, srv1412599966=1, srv288570378=15, srv59733377=21, srv45770472=18, srv1447522566=2, srv1332688651=0, srv1494514965=4, srv2130867416=13, srv926957575=28, srv681991050=22, srv1550799395=5, srv1739629051=7, srv505577208=20, srv771491520=26, srv932428476=29, srv2147088810=14} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-02 21:54:49,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-02 21:54:49,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-02 21:54:49,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-02 21:54:49,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-02 21:54:49,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-02 21:54:49,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-02 21:54:49,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-02 21:54:49,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-02 21:54:49,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-02 21:54:49,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-02 21:54:49,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-02 21:54:49,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-02 21:54:49,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-02 21:54:49,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-02 21:54:49,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-02 21:54:49,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-02 21:54:49,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-02 21:54:49,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-02 21:54:49,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-02 21:54:49,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-02 21:54:49,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-02 21:54:49,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-02 21:54:49,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-02 21:54:49,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-02 21:54:49,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-02 21:54:49,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-02 21:54:49,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-02 21:54:49,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-02 21:54:49,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-02 21:54:49,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-02 21:54:49,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-11-02 21:54:49,552 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-02 21:54:49,552 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1332688651:90 , srv1412599966:90 , srv1447522566:90 , srv1478956867:90 , srv1494514965:90 , srv1550799395:90 , srv1682469068:90 , srv1739629051:90 , srv1830397736:90 , srv197079212:90 , srv1975069627:90 , srv2060591026:90 , srv2077673903:90 , srv2130867416:90 , srv2147088810:90 , srv288570378:90 , srv418378136:90 , srv436594080:90 , srv45770472:90 , srv47489465:90 , srv505577208:90 , srv59733377:90 , srv681991050:90 , srv683282677:90 , srv68810782:90 , srv758966085:90 , srv771491520:90 , srv775437376:90 , srv926957575:90 , srv932428476:90 }
2023-11-02 21:54:49,563 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=202 (was 364), ProcessCount=174 (was 175), AvailableMemoryMB=843 (was 1604)