2024-02-04 19:53:10,262 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2024-02-04 19:53:10,556 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2024-02-04 19:53:10,660 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2024-02-04 19:53:10,724 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-02-04 19:53:10,731 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-02-04 19:53:10,745 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=305, ProcessCount=168, AvailableMemoryMB=3150
2024-02-04 19:53:10,747 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-02-04 19:53:10,747 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-02-04 19:53:10,886 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1005456092:84 , srv1032862408:84 , srv1076517605:84 , srv1082237724:84 , srv1098953972:84 , srv1132094094:84 , srv1163646851:84 , srv1212279381:84 , srv1213113269:84 , srv1213743231:84 , srv1218555354:84 , srv1238272463:84 , srv124184945:84 , srv1249964249:84 , srv12538978:84 , srv1274246403:84 , srv1289903084:84 , srv1302196801:84 , srv1333747698:84 , srv1362646491:84 , srv1368046539:84 , srv1375511316:84 , srv13902331:84 , srv1399730236:84 , srv1405712787:84 , srv1442079598:84 , srv1459004040:84 , srv1519589761:84 , srv1537132568:84 , srv1568951186:84 , srv1570490020:84 , srv1571426524:84 , srv157444674:84 , srv1588870531:84 , srv1593532725:84 , srv1669416539:84 , srv1674017746:84 , srv1692617870:84 , srv1709026965:84 , srv1726539884:84 , srv1738760314:84 , srv1759881130:84 , srv1778355197:84 , srv1801406504:84 , srv180830820:84 , srv1975724219:84 , srv2012728067:84 , srv203080753:84 , srv2031724958:84 , srv2060062430:84 , srv2070069916:84 , srv2076515261:84 , srv2086333527:84 , srv2101751006:84 , srv2110337982:84 , srv2115844579:84 , srv240776984:84 , srv252661560:84 , srv254105722:84 , srv279567172:84 , srv293429091:84 , srv298516002:84 , srv307270360:84 , srv313548538:84 , srv316075896:84 , srv317719923:84 , srv332846821:84 , srv354865752:84 , srv397372999:84 , srv432926130:84 , srv445412730:84 , srv507429626:84 , srv513690103:84 , srv531443410:84 , srv547153974:84 , srv551014491:84 , srv570812977:84 , srv575862091:84 , srv580132628:84 , srv601183459:84 , srv619303459:84 , srv657098906:84 , srv672172881:84 , srv783108583:84 , srv795097099:84 , srv801830680:84 , srv832411606:84 , srv854340963:84 , srv864947182:84 , srv865771048:84 , srv88320294:84 , srv898260838:84 , srv912098132:84 , srv934002937:84 , srv952943396:84 , srv953220300:84 , srv959189749:84 , srv960295165:84 , srv967831955:84 , srv363520183:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2024-02-04 19:53:10,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-04 19:53:10,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1399730236=23, srv2101751006=53, srv575862091=78, srv1005456092=0, srv1709026965=38, srv1082237724=3, srv88320294=91, srv1442079598=25, srv2110337982=54, srv1132094094=5, srv1249964249=13, srv1289903084=16, srv1588870531=33, srv354865752=67, srv1098953972=4, srv1218555354=10, srv397372999=69, srv1459004040=26, srv1593532725=34, srv1801406504=43, srv854340963=88, srv363520183=68, srv2086333527=52, srv547153974=75, srv1238272463=11, srv252661560=57, srv967831955=99, srv1212279381=7, srv1375511316=21, srv298516002=61, srv1213113269=8, srv1975724219=45, srv1537132568=28, srv580132628=79, srv1302196801=17, srv795097099=85, srv1519589761=27, srv1568951186=29, srv124184945=12, srv1669416539=35, srv445412730=71, srv801830680=86, srv2076515261=51, srv507429626=72, srv1778355197=42, srv432926130=70, srv13902331=22, srv313548538=63, srv1738760314=40, srv1032862408=1, srv1692617870=37, srv960295165=98, srv1362646491=19, srv1726539884=39, srv570812977=77, srv783108583=84, srv672172881=83, srv952943396=95, srv2115844579=55, srv953220300=96, srv1405712787=24, srv317719923=65, srv1570490020=30, srv180830820=44, srv934002937=94, srv657098906=82, srv2031724958=48, srv864947182=89, srv1759881130=41, srv2012728067=46, srv1213743231=9, srv332846821=66, srv1274246403=15, srv157444674=32, srv1076517605=2, srv513690103=73, srv1333747698=18, srv1368046539=20, srv240776984=56, srv203080753=47, srv601183459=80, srv2070069916=50, srv293429091=60, srv912098132=93, srv1571426524=31, srv279567172=59, srv316075896=64, srv551014491=76, srv254105722=58, srv1674017746=36, srv307270360=62, srv2060062430=49, srv619303459=81, srv898260838=92, srv1163646851=6, srv531443410=74, srv832411606=87, srv865771048=90, srv959189749=97, srv12538978=14} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-04 19:53:10,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-04 19:53:10,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-04 19:53:10,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-04 19:53:10,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-04 19:53:10,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-04 19:53:10,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-04 19:53:10,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-04 19:53:10,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-04 19:53:11,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-04 19:53:11,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-04 19:53:11,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-04 19:53:11,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-04 19:53:11,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-04 19:53:11,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-04 19:53:11,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-04 19:53:11,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-04 19:53:11,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-04 19:53:11,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-04 19:53:11,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-04 19:53:11,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-04 19:53:11,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-04 19:53:11,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-04 19:53:11,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-04 19:53:11,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-04 19:53:11,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-04 19:53:11,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-04 19:53:11,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-04 19:53:11,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-04 19:53:11,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-04 19:53:11,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-04 19:53:11,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-02-04 19:53:11,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-02-04 19:53:11,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-02-04 19:53:11,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-02-04 19:53:11,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-02-04 19:53:11,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-02-04 19:53:11,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-02-04 19:53:11,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-02-04 19:53:11,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-02-04 19:53:11,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-02-04 19:53:11,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-02-04 19:53:11,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-02-04 19:53:11,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-02-04 19:53:11,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-02-04 19:53:11,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-02-04 19:53:11,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-02-04 19:53:11,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-02-04 19:53:11,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-02-04 19:53:11,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-02-04 19:53:11,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-02-04 19:53:11,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-02-04 19:53:11,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-02-04 19:53:11,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-02-04 19:53:11,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-02-04 19:53:11,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-02-04 19:53:11,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-02-04 19:53:11,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-02-04 19:53:11,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-02-04 19:53:11,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-02-04 19:53:11,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-02-04 19:53:11,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-02-04 19:53:11,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-02-04 19:53:11,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-02-04 19:53:11,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-02-04 19:53:11,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-02-04 19:53:11,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-02-04 19:53:11,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-02-04 19:53:11,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-02-04 19:53:11,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-02-04 19:53:11,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-02-04 19:53:11,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-02-04 19:53:11,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-02-04 19:53:11,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-02-04 19:53:11,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-02-04 19:53:11,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-02-04 19:53:11,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-02-04 19:53:11,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-02-04 19:53:11,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-02-04 19:53:11,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-02-04 19:53:11,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-02-04 19:53:11,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-02-04 19:53:11,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-02-04 19:53:11,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-02-04 19:53:11,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-02-04 19:53:11,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-02-04 19:53:11,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-02-04 19:53:11,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-02-04 19:53:11,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-02-04 19:53:11,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-02-04 19:53:11,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-02-04 19:53:11,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-02-04 19:53:11,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-02-04 19:53:11,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-02-04 19:53:11,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-02-04 19:53:11,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-02-04 19:53:11,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-02-04 19:53:11,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-02-04 19:53:11,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-02-04 19:53:11,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-02-04 19:53:11,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-02-04 19:53:11,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-02-04 19:53:11,268 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-04 19:53:11,269 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-02-04 19:53:11,269 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-02-04 19:53:11,741 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 703 ms to try 5000 different iterations.  Found a solution that moves 4741 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17231585143420103. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03940415419092487, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05445165897365446, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5267777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03940415419092487, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.16741666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23466666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-04 19:53:11,757 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv864947182:81 , srv1218555354:82 , srv1669416539:82 , srv801830680:82 , srv854340963:82 , srv912098132:82 , srv1032862408:83 , srv1212279381:83 , srv1302196801:83 , srv1459004040:83 , srv1568951186:83 , srv1975724219:83 , srv252661560:83 , srv279567172:83 , srv317719923:83 , srv445412730:83 , srv88320294:83 , srv1571426524:84 , srv1593532725:84 , srv1759881130:84 , srv203080753:84 , srv2110337982:84 , srv832411606:84 , srv865771048:84 , srv959189749:84 , srv960295165:84 , srv1213113269:85 , srv12538978:85 , srv1405712787:85 , srv1778355197:85 , srv513690103:85 , srv795097099:85 , srv1076517605:86 , srv1132094094:86 , srv1238272463:86 , srv124184945:86 , srv1333747698:86 , srv1375511316:86 , srv13902331:86 , srv1442079598:86 , srv1674017746:86 , srv1692617870:86 , srv1726539884:86 , srv2012728067:86 , srv2031724958:86 , srv2070069916:86 , srv240776984:86 , srv313548538:86 , srv316075896:86 , srv332846821:86 , srv354865752:86 , srv570812977:86 , srv967831955:86 , srv1098953972:87 , srv1249964249:87 , srv1274246403:87 , srv1399730236:87 , srv1570490020:87 , srv157444674:87 , srv1709026965:87 , srv2115844579:87 , srv254105722:87 , srv298516002:87 , srv397372999:87 , srv507429626:87 , srv547153974:87 , srv575862091:87 , srv580132628:87 , srv601183459:87 , srv657098906:87 , srv672172881:87 , srv1005456092:88 , srv1082237724:88 , srv1213743231:88 , srv1801406504:88 , srv2060062430:88 , srv293429091:88 , srv307270360:88 , srv898260838:88 , srv1588870531:89 , srv180830820:89 , srv2101751006:89 , srv432926130:89 , srv952943396:89 , srv1163646851:90 , srv1362646491:90 , srv1519589761:90 , srv2076515261:90 , srv531443410:90 , srv619303459:90 , srv783108583:90 , srv953220300:90 , srv551014491:91 , srv1537132568:92 , srv2086333527:92 , srv934002937:92 , srv1289903084:94 , srv1368046539:94 , srv1738760314:94 , srv363520183:440 }
2024-02-04 19:53:11,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-04 19:53:11,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1399730236=23, srv2101751006=53, srv575862091=78, srv1005456092=0, srv1709026965=38, srv1082237724=3, srv88320294=91, srv1442079598=25, srv2110337982=54, srv1132094094=5, srv1249964249=13, srv1289903084=16, srv1588870531=33, srv354865752=67, srv1098953972=4, srv1218555354=10, srv397372999=69, srv1459004040=26, srv1593532725=34, srv1801406504=43, srv854340963=88, srv363520183=68, srv2086333527=52, srv547153974=75, srv1238272463=11, srv252661560=57, srv967831955=99, srv1212279381=7, srv1375511316=21, srv298516002=61, srv1213113269=8, srv1975724219=45, srv1537132568=28, srv580132628=79, srv1302196801=17, srv795097099=85, srv1519589761=27, srv1568951186=29, srv124184945=12, srv1669416539=35, srv445412730=71, srv801830680=86, srv2076515261=51, srv507429626=72, srv1778355197=42, srv432926130=70, srv13902331=22, srv313548538=63, srv1738760314=40, srv1032862408=1, srv1692617870=37, srv960295165=98, srv1362646491=19, srv1726539884=39, srv570812977=77, srv783108583=84, srv672172881=83, srv952943396=95, srv2115844579=55, srv953220300=96, srv1405712787=24, srv317719923=65, srv1570490020=30, srv180830820=44, srv934002937=94, srv657098906=82, srv2031724958=48, srv864947182=89, srv1759881130=41, srv2012728067=46, srv1213743231=9, srv332846821=66, srv1274246403=15, srv157444674=32, srv1076517605=2, srv513690103=73, srv1333747698=18, srv1368046539=20, srv240776984=56, srv203080753=47, srv601183459=80, srv2070069916=50, srv293429091=60, srv912098132=93, srv1571426524=31, srv279567172=59, srv316075896=64, srv551014491=76, srv254105722=58, srv1674017746=36, srv307270360=62, srv2060062430=49, srv619303459=81, srv898260838=92, srv1163646851=6, srv531443410=74, srv832411606=87, srv865771048=90, srv959189749=97, srv12538978=14} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-04 19:53:11,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-04 19:53:11,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-04 19:53:11,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-04 19:53:11,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-04 19:53:11,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-04 19:53:11,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-04 19:53:11,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-04 19:53:11,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-04 19:53:11,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-04 19:53:11,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-04 19:53:11,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-04 19:53:11,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-04 19:53:11,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-04 19:53:11,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-04 19:53:11,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-04 19:53:11,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-04 19:53:11,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-04 19:53:11,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-04 19:53:11,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-04 19:53:11,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-04 19:53:11,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-04 19:53:11,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-04 19:53:11,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-04 19:53:11,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-04 19:53:11,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-04 19:53:11,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-04 19:53:11,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-04 19:53:11,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-04 19:53:11,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-04 19:53:11,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-04 19:53:11,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-02-04 19:53:11,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-02-04 19:53:11,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-02-04 19:53:11,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-02-04 19:53:11,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-02-04 19:53:11,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-02-04 19:53:11,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-02-04 19:53:11,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-02-04 19:53:11,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-02-04 19:53:11,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-02-04 19:53:11,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-02-04 19:53:11,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-02-04 19:53:11,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-02-04 19:53:11,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-02-04 19:53:11,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-02-04 19:53:11,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-02-04 19:53:11,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-02-04 19:53:11,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-02-04 19:53:11,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-02-04 19:53:11,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-02-04 19:53:11,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-02-04 19:53:11,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-02-04 19:53:11,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-02-04 19:53:11,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-02-04 19:53:11,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-02-04 19:53:11,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-02-04 19:53:11,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-02-04 19:53:11,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-02-04 19:53:11,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-02-04 19:53:11,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-02-04 19:53:11,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-02-04 19:53:11,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-02-04 19:53:11,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-02-04 19:53:11,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-02-04 19:53:11,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-02-04 19:53:11,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-02-04 19:53:11,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-02-04 19:53:11,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-02-04 19:53:11,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-02-04 19:53:11,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-02-04 19:53:11,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-02-04 19:53:11,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-02-04 19:53:11,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-02-04 19:53:11,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-02-04 19:53:11,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-02-04 19:53:11,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-02-04 19:53:11,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-02-04 19:53:11,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-02-04 19:53:11,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-02-04 19:53:11,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-02-04 19:53:11,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-02-04 19:53:11,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-02-04 19:53:11,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-02-04 19:53:11,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-02-04 19:53:11,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-02-04 19:53:11,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-02-04 19:53:11,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-02-04 19:53:11,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-02-04 19:53:11,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-02-04 19:53:11,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-02-04 19:53:11,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-02-04 19:53:11,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-02-04 19:53:11,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-02-04 19:53:11,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-02-04 19:53:11,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-02-04 19:53:11,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-02-04 19:53:11,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-02-04 19:53:11,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-02-04 19:53:11,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-02-04 19:53:11,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-02-04 19:53:11,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-02-04 19:53:12,024 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03940415419092487, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05445165897365446, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03940415419092487, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.16741666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23466666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-04 19:53:12,024 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-02-04 19:53:12,024 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17228265424266323, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03940415419092487, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05445165897365446, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03940415419092487, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.16741666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23466666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-02-04 19:53:12,313 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 482 ms to try 5000 different iterations.  Found a solution that moves 2613 regions; Going from a computed imbalance of 0.17228265424266323 to a new imbalance of 0.013550893687950089. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022168284710107875); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.030933838379715455, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29033333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022168284710107875); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010333333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04425, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-04 19:53:12,321 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1289903084:78 , srv1368046539:79 , srv1362646491:80 , srv1537132568:80 , srv1005456092:81 , srv1738760314:81 , srv2086333527:81 , srv531443410:81 , srv551014491:81 , srv934002937:81 , srv953220300:81 , srv1163646851:82 , srv1692617870:82 , srv1801406504:82 , srv783108583:82 , srv898260838:82 , srv2076515261:83 , srv1213743231:84 , srv1249964249:84 , srv157444674:84 , srv1588870531:84 , srv2115844579:84 , srv240776984:84 , srv293429091:84 , srv397372999:84 , srv619303459:84 , srv657098906:84 , srv672172881:84 , srv1076517605:85 , srv1082237724:85 , srv1098953972:85 , srv1132094094:85 , srv1238272463:85 , srv124184945:85 , srv1333747698:85 , srv13902331:85 , srv1442079598:85 , srv1726539884:85 , srv1759881130:85 , srv180830820:85 , srv2101751006:85 , srv513690103:85 , srv570812977:85 , srv952943396:85 , srv1213113269:86 , srv1375511316:86 , srv1399730236:86 , srv1519589761:86 , srv1709026965:86 , srv2060062430:86 , srv254105722:86 , srv307270360:86 , srv332846821:86 , srv354865752:86 , srv432926130:86 , srv445412730:86 , srv547153974:86 , srv575862091:86 , srv580132628:86 , srv967831955:86 , srv1274246403:87 , srv1302196801:87 , srv1405712787:87 , srv1570490020:87 , srv1571426524:87 , srv1593532725:87 , srv1674017746:87 , srv1778355197:87 , srv2012728067:87 , srv2031724958:87 , srv252661560:87 , srv298516002:87 , srv313548538:87 , srv316075896:87 , srv507429626:87 , srv601183459:87 , srv795097099:87 , srv832411606:87 , srv865771048:87 , srv959189749:87 , srv1212279381:88 , srv12538978:88 , srv1669416539:88 , srv1975724219:88 , srv203080753:88 , srv2070069916:88 , srv2110337982:88 , srv279567172:88 , srv1032862408:89 , srv1568951186:89 , srv317719923:89 , srv88320294:89 , srv912098132:89 , srv960295165:89 , srv1218555354:90 , srv1459004040:90 , srv801830680:91 , srv854340963:91 , srv864947182:92 , srv363520183:531 }
2024-02-04 19:53:12,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-04 19:53:12,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1399730236=23, srv2101751006=53, srv575862091=78, srv1005456092=0, srv1709026965=38, srv1082237724=3, srv88320294=91, srv1442079598=25, srv2110337982=54, srv1132094094=5, srv1249964249=13, srv1289903084=16, srv1588870531=33, srv354865752=67, srv1098953972=4, srv1218555354=10, srv397372999=69, srv1459004040=26, srv1593532725=34, srv1801406504=43, srv854340963=88, srv363520183=68, srv2086333527=52, srv547153974=75, srv1238272463=11, srv252661560=57, srv967831955=99, srv1212279381=7, srv1375511316=21, srv298516002=61, srv1213113269=8, srv1975724219=45, srv1537132568=28, srv580132628=79, srv1302196801=17, srv795097099=85, srv1519589761=27, srv1568951186=29, srv124184945=12, srv1669416539=35, srv445412730=71, srv801830680=86, srv2076515261=51, srv507429626=72, srv1778355197=42, srv432926130=70, srv13902331=22, srv313548538=63, srv1738760314=40, srv1032862408=1, srv1692617870=37, srv960295165=98, srv1362646491=19, srv1726539884=39, srv570812977=77, srv783108583=84, srv672172881=83, srv952943396=95, srv2115844579=55, srv953220300=96, srv1405712787=24, srv317719923=65, srv1570490020=30, srv180830820=44, srv934002937=94, srv657098906=82, srv2031724958=48, srv864947182=89, srv1759881130=41, srv2012728067=46, srv1213743231=9, srv332846821=66, srv1274246403=15, srv157444674=32, srv1076517605=2, srv513690103=73, srv1333747698=18, srv1368046539=20, srv240776984=56, srv203080753=47, srv601183459=80, srv2070069916=50, srv293429091=60, srv912098132=93, srv1571426524=31, srv279567172=59, srv316075896=64, srv551014491=76, srv254105722=58, srv1674017746=36, srv307270360=62, srv2060062430=49, srv619303459=81, srv898260838=92, srv1163646851=6, srv531443410=74, srv832411606=87, srv865771048=90, srv959189749=97, srv12538978=14} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-04 19:53:12,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-04 19:53:12,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-04 19:53:12,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-04 19:53:12,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-04 19:53:12,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-04 19:53:12,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-04 19:53:12,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-04 19:53:12,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-04 19:53:12,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-04 19:53:12,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-04 19:53:12,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-04 19:53:12,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-04 19:53:12,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-04 19:53:12,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-04 19:53:12,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-04 19:53:12,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-04 19:53:12,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-04 19:53:12,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-04 19:53:12,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-04 19:53:12,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-04 19:53:12,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-04 19:53:12,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-04 19:53:12,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-04 19:53:12,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-04 19:53:12,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-04 19:53:12,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-04 19:53:12,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-04 19:53:12,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-04 19:53:12,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-04 19:53:12,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-04 19:53:12,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-02-04 19:53:12,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-02-04 19:53:12,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-02-04 19:53:12,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-02-04 19:53:12,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-02-04 19:53:12,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-02-04 19:53:12,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-02-04 19:53:12,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-02-04 19:53:12,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-02-04 19:53:12,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-02-04 19:53:12,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-02-04 19:53:12,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-02-04 19:53:12,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-02-04 19:53:12,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-02-04 19:53:12,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-02-04 19:53:12,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-02-04 19:53:12,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-02-04 19:53:12,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-02-04 19:53:12,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-02-04 19:53:12,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-02-04 19:53:12,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-02-04 19:53:12,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-02-04 19:53:12,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-02-04 19:53:12,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-02-04 19:53:12,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-02-04 19:53:12,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-02-04 19:53:12,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-02-04 19:53:12,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-02-04 19:53:12,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-02-04 19:53:12,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-02-04 19:53:12,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-02-04 19:53:12,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-02-04 19:53:12,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-02-04 19:53:12,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-02-04 19:53:12,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-02-04 19:53:12,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-02-04 19:53:12,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-02-04 19:53:12,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-02-04 19:53:12,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-02-04 19:53:12,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-02-04 19:53:12,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-02-04 19:53:12,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-02-04 19:53:12,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-02-04 19:53:12,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-02-04 19:53:12,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-02-04 19:53:12,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-02-04 19:53:12,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-02-04 19:53:12,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-02-04 19:53:12,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-02-04 19:53:12,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-02-04 19:53:12,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-02-04 19:53:12,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-02-04 19:53:12,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-02-04 19:53:12,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-02-04 19:53:12,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-02-04 19:53:12,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-02-04 19:53:12,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-02-04 19:53:12,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-02-04 19:53:12,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-02-04 19:53:12,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-02-04 19:53:12,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-02-04 19:53:12,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-02-04 19:53:12,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-02-04 19:53:12,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-02-04 19:53:12,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-02-04 19:53:12,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-02-04 19:53:12,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-02-04 19:53:12,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-02-04 19:53:12,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-02-04 19:53:12,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-02-04 19:53:12,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-02-04 19:53:12,478 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022168284710107875); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.030933838379715455, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022168284710107875); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010333333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04425, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-04 19:53:12,478 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-02-04 19:53:12,479 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.013532597070888651, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022168284710107875); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.030933838379715455, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022168284710107875); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010333333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04425, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-02-04 19:53:12,870 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 508 ms to try 5000 different iterations.  Found a solution that moves 929 regions; Going from a computed imbalance of 0.013532597070888651 to a new imbalance of 8.518444078433501E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00836264951860019); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.012180829817351595); MoveCostFunction : (multiplier=7.0, imbalance=0.10322222222222223, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.00836264951860019); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.005); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-04 19:53:12,872 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1519589761:80 , srv1738760314:82 , srv2070069916:82 , srv12538978:83 , srv1399730236:83 , srv1570490020:83 , srv1588870531:83 , srv2060062430:83 , srv307270360:83 , srv432926130:83 , srv854340963:83 , srv1032862408:84 , srv1082237724:84 , srv1274246403:84 , srv1368046539:84 , srv1674017746:84 , srv180830820:84 , srv2012728067:84 , srv203080753:84 , srv2076515261:84 , srv2101751006:84 , srv254105722:84 , srv313548538:84 , srv354865752:84 , srv507429626:84 , srv547153974:84 , srv580132628:84 , srv601183459:84 , srv619303459:84 , srv795097099:84 , srv801830680:84 , srv864947182:84 , srv934002937:84 , srv952943396:84 , srv1163646851:85 , srv1212279381:85 , srv1213743231:85 , srv1375511316:85 , srv1459004040:85 , srv1568951186:85 , srv1709026965:85 , srv1778355197:85 , srv2031724958:85 , srv2086333527:85 , srv2115844579:85 , srv293429091:85 , srv298516002:85 , srv316075896:85 , srv317719923:85 , srv575862091:85 , srv657098906:85 , srv832411606:85 , srv88320294:85 , srv959189749:85 , srv960295165:85 , srv1076517605:86 , srv1098953972:86 , srv1218555354:86 , srv1238272463:86 , srv124184945:86 , srv1333747698:86 , srv13902331:86 , srv1405712787:86 , srv1442079598:86 , srv1571426524:86 , srv1593532725:86 , srv1669416539:86 , srv2110337982:86 , srv279567172:86 , srv332846821:86 , srv397372999:86 , srv551014491:86 , srv783108583:86 , srv865771048:86 , srv898260838:86 , srv967831955:86 , srv1132094094:87 , srv1213113269:87 , srv1249964249:87 , srv1289903084:87 , srv1302196801:87 , srv1362646491:87 , srv157444674:87 , srv1726539884:87 , srv1759881130:87 , srv1801406504:87 , srv1975724219:87 , srv240776984:87 , srv252661560:87 , srv445412730:87 , srv513690103:87 , srv570812977:87 , srv912098132:87 , srv953220300:87 , srv1005456092:88 , srv531443410:88 , srv672172881:88 , srv1537132568:89 , srv1692617870:89 , srv363520183:561 }
2024-02-04 19:53:12,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-04 19:53:12,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1399730236=23, srv2101751006=53, srv575862091=78, srv1005456092=0, srv1709026965=38, srv1082237724=3, srv88320294=91, srv1442079598=25, srv2110337982=54, srv1132094094=5, srv1249964249=13, srv1289903084=16, srv1588870531=33, srv354865752=67, srv1098953972=4, srv1218555354=10, srv397372999=69, srv1459004040=26, srv1593532725=34, srv1801406504=43, srv854340963=88, srv363520183=68, srv2086333527=52, srv547153974=75, srv1238272463=11, srv252661560=57, srv967831955=99, srv1212279381=7, srv1375511316=21, srv298516002=61, srv1213113269=8, srv1975724219=45, srv1537132568=28, srv580132628=79, srv1302196801=17, srv795097099=85, srv1519589761=27, srv1568951186=29, srv124184945=12, srv1669416539=35, srv445412730=71, srv801830680=86, srv2076515261=51, srv507429626=72, srv1778355197=42, srv432926130=70, srv13902331=22, srv313548538=63, srv1738760314=40, srv1032862408=1, srv1692617870=37, srv960295165=98, srv1362646491=19, srv1726539884=39, srv570812977=77, srv783108583=84, srv672172881=83, srv952943396=95, srv2115844579=55, srv953220300=96, srv1405712787=24, srv317719923=65, srv1570490020=30, srv180830820=44, srv934002937=94, srv657098906=82, srv2031724958=48, srv864947182=89, srv1759881130=41, srv2012728067=46, srv1213743231=9, srv332846821=66, srv1274246403=15, srv157444674=32, srv1076517605=2, srv513690103=73, srv1333747698=18, srv1368046539=20, srv240776984=56, srv203080753=47, srv601183459=80, srv2070069916=50, srv293429091=60, srv912098132=93, srv1571426524=31, srv279567172=59, srv316075896=64, srv551014491=76, srv254105722=58, srv1674017746=36, srv307270360=62, srv2060062430=49, srv619303459=81, srv898260838=92, srv1163646851=6, srv531443410=74, srv832411606=87, srv865771048=90, srv959189749=97, srv12538978=14} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-04 19:53:12,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-04 19:53:12,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-04 19:53:12,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-04 19:53:12,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-04 19:53:12,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-04 19:53:12,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-04 19:53:12,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-04 19:53:12,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-04 19:53:12,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-04 19:53:12,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-04 19:53:12,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-04 19:53:12,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-04 19:53:12,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-04 19:53:12,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-04 19:53:12,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-04 19:53:12,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-04 19:53:12,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-04 19:53:12,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-04 19:53:12,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-04 19:53:12,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-04 19:53:12,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-04 19:53:12,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-04 19:53:12,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-04 19:53:12,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-04 19:53:12,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-04 19:53:12,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-04 19:53:12,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-04 19:53:12,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-04 19:53:12,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-04 19:53:12,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-04 19:53:12,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-02-04 19:53:12,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-02-04 19:53:12,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-02-04 19:53:12,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-02-04 19:53:12,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-02-04 19:53:12,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-02-04 19:53:12,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-02-04 19:53:12,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-02-04 19:53:12,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-02-04 19:53:12,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-02-04 19:53:12,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-02-04 19:53:12,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-02-04 19:53:12,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-02-04 19:53:12,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-02-04 19:53:12,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-02-04 19:53:12,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-02-04 19:53:12,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-02-04 19:53:12,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-02-04 19:53:12,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-02-04 19:53:12,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-02-04 19:53:12,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-02-04 19:53:12,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-02-04 19:53:12,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-02-04 19:53:12,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-02-04 19:53:12,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-02-04 19:53:12,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-02-04 19:53:12,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-02-04 19:53:12,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-02-04 19:53:12,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-02-04 19:53:12,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-02-04 19:53:12,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-02-04 19:53:12,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-02-04 19:53:12,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-02-04 19:53:12,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-02-04 19:53:12,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-02-04 19:53:12,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-02-04 19:53:12,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-02-04 19:53:12,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-02-04 19:53:12,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-02-04 19:53:12,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-02-04 19:53:12,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-02-04 19:53:12,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-02-04 19:53:12,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-02-04 19:53:12,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-02-04 19:53:12,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-02-04 19:53:12,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-02-04 19:53:12,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-02-04 19:53:12,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-02-04 19:53:12,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-02-04 19:53:12,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-02-04 19:53:12,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-02-04 19:53:12,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-02-04 19:53:12,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-02-04 19:53:12,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-02-04 19:53:12,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-02-04 19:53:12,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-02-04 19:53:12,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-02-04 19:53:12,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-02-04 19:53:12,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-02-04 19:53:12,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-02-04 19:53:12,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-02-04 19:53:12,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-02-04 19:53:12,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-02-04 19:53:12,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-02-04 19:53:12,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-02-04 19:53:12,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-02-04 19:53:12,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-02-04 19:53:12,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-02-04 19:53:12,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-02-04 19:53:12,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-02-04 19:53:12,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-02-04 19:53:13,023 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00836264951860019); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.012180829817351595); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.00836264951860019); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.005); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-04 19:53:13,023 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-02-04 19:53:13,024 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=8.453394108092606E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00836264951860019); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.012180829817351595); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.00836264951860019); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.005); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-02-04 19:53:13,362 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 452 ms to try 5000 different iterations.  Found a solution that moves 335 regions; Going from a computed imbalance of 8.453394108092606E-4 to a new imbalance of 5.999281113204734E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0022721660362024225); MoveCostFunction : (multiplier=7.0, imbalance=0.03722222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=5.0E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-04 19:53:13,364 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1537132568:80 , srv1289903084:83 , srv531443410:83 , srv672172881:83 , srv1098953972:84 , srv1132094094:84 , srv1213113269:84 , srv1213743231:84 , srv1218555354:84 , srv1249964249:84 , srv1274246403:84 , srv1405712787:84 , srv157444674:84 , srv1709026965:84 , srv1726539884:84 , srv180830820:84 , srv1975724219:84 , srv2012728067:84 , srv2031724958:84 , srv2086333527:84 , srv2101751006:84 , srv2110337982:84 , srv240776984:84 , srv279567172:84 , srv298516002:84 , srv316075896:84 , srv317719923:84 , srv332846821:84 , srv432926130:84 , srv551014491:84 , srv570812977:84 , srv575862091:84 , srv601183459:84 , srv619303459:84 , srv783108583:84 , srv912098132:84 , srv952943396:84 , srv953220300:84 , srv960295165:84 , srv1005456092:85 , srv1032862408:85 , srv1076517605:85 , srv1082237724:85 , srv1238272463:85 , srv124184945:85 , srv1302196801:85 , srv1362646491:85 , srv1368046539:85 , srv1375511316:85 , srv13902331:85 , srv1442079598:85 , srv1459004040:85 , srv1568951186:85 , srv1570490020:85 , srv1571426524:85 , srv1588870531:85 , srv1593532725:85 , srv1674017746:85 , srv1692617870:85 , srv1738760314:85 , srv1778355197:85 , srv203080753:85 , srv2060062430:85 , srv2076515261:85 , srv2115844579:85 , srv254105722:85 , srv293429091:85 , srv307270360:85 , srv313548538:85 , srv397372999:85 , srv507429626:85 , srv513690103:85 , srv547153974:85 , srv580132628:85 , srv801830680:85 , srv864947182:85 , srv865771048:85 , srv88320294:85 , srv967831955:85 , srv1163646851:86 , srv1212279381:86 , srv12538978:86 , srv1333747698:86 , srv1399730236:86 , srv1519589761:86 , srv1669416539:86 , srv1759881130:86 , srv1801406504:86 , srv2070069916:86 , srv252661560:86 , srv445412730:86 , srv657098906:86 , srv795097099:86 , srv832411606:86 , srv854340963:86 , srv898260838:86 , srv934002937:86 , srv959189749:86 , srv354865752:87 , srv363520183:610 }
2024-02-04 19:53:13,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-04 19:53:13,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1399730236=23, srv2101751006=53, srv575862091=78, srv1005456092=0, srv1709026965=38, srv1082237724=3, srv88320294=91, srv1442079598=25, srv2110337982=54, srv1132094094=5, srv1249964249=13, srv1289903084=16, srv1588870531=33, srv354865752=67, srv1098953972=4, srv1218555354=10, srv397372999=69, srv1459004040=26, srv1593532725=34, srv1801406504=43, srv854340963=88, srv363520183=68, srv2086333527=52, srv547153974=75, srv1238272463=11, srv252661560=57, srv967831955=99, srv1212279381=7, srv1375511316=21, srv298516002=61, srv1213113269=8, srv1975724219=45, srv1537132568=28, srv580132628=79, srv1302196801=17, srv795097099=85, srv1519589761=27, srv1568951186=29, srv124184945=12, srv1669416539=35, srv445412730=71, srv801830680=86, srv2076515261=51, srv507429626=72, srv1778355197=42, srv432926130=70, srv13902331=22, srv313548538=63, srv1738760314=40, srv1032862408=1, srv1692617870=37, srv960295165=98, srv1362646491=19, srv1726539884=39, srv570812977=77, srv783108583=84, srv672172881=83, srv952943396=95, srv2115844579=55, srv953220300=96, srv1405712787=24, srv317719923=65, srv1570490020=30, srv180830820=44, srv934002937=94, srv657098906=82, srv2031724958=48, srv864947182=89, srv1759881130=41, srv2012728067=46, srv1213743231=9, srv332846821=66, srv1274246403=15, srv157444674=32, srv1076517605=2, srv513690103=73, srv1333747698=18, srv1368046539=20, srv240776984=56, srv203080753=47, srv601183459=80, srv2070069916=50, srv293429091=60, srv912098132=93, srv1571426524=31, srv279567172=59, srv316075896=64, srv551014491=76, srv254105722=58, srv1674017746=36, srv307270360=62, srv2060062430=49, srv619303459=81, srv898260838=92, srv1163646851=6, srv531443410=74, srv832411606=87, srv865771048=90, srv959189749=97, srv12538978=14} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-04 19:53:13,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-04 19:53:13,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-04 19:53:13,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-04 19:53:13,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-04 19:53:13,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-04 19:53:13,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-04 19:53:13,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-04 19:53:13,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-04 19:53:13,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-04 19:53:13,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-04 19:53:13,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-04 19:53:13,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-04 19:53:13,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-04 19:53:13,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-04 19:53:13,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-04 19:53:13,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-04 19:53:13,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-04 19:53:13,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-04 19:53:13,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-04 19:53:13,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-04 19:53:13,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-04 19:53:13,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-04 19:53:13,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-04 19:53:13,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-04 19:53:13,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-04 19:53:13,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-04 19:53:13,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-04 19:53:13,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-04 19:53:13,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-04 19:53:13,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-04 19:53:13,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-02-04 19:53:13,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-02-04 19:53:13,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-02-04 19:53:13,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-02-04 19:53:13,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-02-04 19:53:13,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-02-04 19:53:13,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-02-04 19:53:13,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-02-04 19:53:13,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-02-04 19:53:13,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-02-04 19:53:13,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-02-04 19:53:13,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-02-04 19:53:13,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-02-04 19:53:13,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-02-04 19:53:13,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-02-04 19:53:13,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-02-04 19:53:13,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-02-04 19:53:13,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-02-04 19:53:13,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-02-04 19:53:13,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-02-04 19:53:13,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-02-04 19:53:13,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-02-04 19:53:13,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-02-04 19:53:13,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-02-04 19:53:13,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-02-04 19:53:13,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-02-04 19:53:13,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-02-04 19:53:13,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-02-04 19:53:13,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-02-04 19:53:13,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-02-04 19:53:13,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-02-04 19:53:13,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-02-04 19:53:13,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-02-04 19:53:13,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-02-04 19:53:13,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-02-04 19:53:13,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-02-04 19:53:13,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-02-04 19:53:13,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-02-04 19:53:13,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-02-04 19:53:13,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-02-04 19:53:13,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-02-04 19:53:13,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-02-04 19:53:13,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-02-04 19:53:13,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-02-04 19:53:13,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-02-04 19:53:13,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-02-04 19:53:13,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-02-04 19:53:13,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-02-04 19:53:13,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-02-04 19:53:13,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-02-04 19:53:13,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-02-04 19:53:13,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-02-04 19:53:13,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-02-04 19:53:13,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-02-04 19:53:13,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-02-04 19:53:13,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-02-04 19:53:13,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-02-04 19:53:13,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-02-04 19:53:13,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-02-04 19:53:13,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-02-04 19:53:13,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-02-04 19:53:13,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-02-04 19:53:13,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-02-04 19:53:13,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-02-04 19:53:13,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-02-04 19:53:13,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-02-04 19:53:13,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-02-04 19:53:13,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-02-04 19:53:13,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-02-04 19:53:13,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-02-04 19:53:13,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-02-04 19:53:13,542 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0022721660362024225); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=5.0E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-04 19:53:13,542 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-02-04 19:53:13,543 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=5.7647090995965556E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0022721660362024225); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=5.0E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-02-04 19:53:13,803 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 396 ms to try 5000 different iterations.  Found a solution that moves 47 regions; Going from a computed imbalance of 5.7647090995965556E-5 to a new imbalance of 7.073170087849392E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014982219165849825); MoveCostFunction : (multiplier=7.0, imbalance=0.005222222222222222); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-04 19:53:13,803 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1163646851:83 , srv1333747698:83 , srv1459004040:83 , srv1801406504:83 , srv252661560:83 , srv298516002:83 , srv354865752:83 , srv507429626:83 , srv934002937:83 , srv967831955:83 , srv1005456092:84 , srv1076517605:84 , srv1082237724:84 , srv1098953972:84 , srv1132094094:84 , srv1212279381:84 , srv1213113269:84 , srv1218555354:84 , srv1238272463:84 , srv124184945:84 , srv1249964249:84 , srv12538978:84 , srv1274246403:84 , srv1289903084:84 , srv1302196801:84 , srv1362646491:84 , srv1368046539:84 , srv1375511316:84 , srv13902331:84 , srv1399730236:84 , srv1405712787:84 , srv1442079598:84 , srv1519589761:84 , srv1568951186:84 , srv1570490020:84 , srv1571426524:84 , srv157444674:84 , srv1593532725:84 , srv1669416539:84 , srv1674017746:84 , srv1692617870:84 , srv1709026965:84 , srv1726539884:84 , srv1738760314:84 , srv1759881130:84 , srv1778355197:84 , srv180830820:84 , srv1975724219:84 , srv2031724958:84 , srv2060062430:84 , srv2070069916:84 , srv2076515261:84 , srv2086333527:84 , srv2101751006:84 , srv2110337982:84 , srv254105722:84 , srv293429091:84 , srv307270360:84 , srv313548538:84 , srv316075896:84 , srv332846821:84 , srv397372999:84 , srv432926130:84 , srv445412730:84 , srv513690103:84 , srv531443410:84 , srv547153974:84 , srv551014491:84 , srv570812977:84 , srv575862091:84 , srv580132628:84 , srv601183459:84 , srv619303459:84 , srv657098906:84 , srv672172881:84 , srv783108583:84 , srv795097099:84 , srv801830680:84 , srv832411606:84 , srv854340963:84 , srv864947182:84 , srv865771048:84 , srv88320294:84 , srv898260838:84 , srv912098132:84 , srv952943396:84 , srv953220300:84 , srv959189749:84 , srv960295165:84 , srv1032862408:85 , srv1213743231:85 , srv1537132568:85 , srv1588870531:85 , srv2012728067:85 , srv203080753:85 , srv2115844579:85 , srv240776984:85 , srv279567172:85 , srv317719923:85 , srv363520183:684 }
2024-02-04 19:53:13,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-04 19:53:13,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1399730236=23, srv2101751006=53, srv575862091=78, srv1005456092=0, srv1709026965=38, srv1082237724=3, srv88320294=91, srv1442079598=25, srv2110337982=54, srv1132094094=5, srv1249964249=13, srv1289903084=16, srv1588870531=33, srv354865752=67, srv1098953972=4, srv1218555354=10, srv397372999=69, srv1459004040=26, srv1593532725=34, srv1801406504=43, srv854340963=88, srv363520183=68, srv2086333527=52, srv547153974=75, srv1238272463=11, srv252661560=57, srv967831955=99, srv1212279381=7, srv1375511316=21, srv298516002=61, srv1213113269=8, srv1975724219=45, srv1537132568=28, srv580132628=79, srv1302196801=17, srv795097099=85, srv1519589761=27, srv1568951186=29, srv124184945=12, srv1669416539=35, srv445412730=71, srv801830680=86, srv2076515261=51, srv507429626=72, srv1778355197=42, srv432926130=70, srv13902331=22, srv313548538=63, srv1738760314=40, srv1032862408=1, srv1692617870=37, srv960295165=98, srv1362646491=19, srv1726539884=39, srv570812977=77, srv783108583=84, srv672172881=83, srv952943396=95, srv2115844579=55, srv953220300=96, srv1405712787=24, srv317719923=65, srv1570490020=30, srv180830820=44, srv934002937=94, srv657098906=82, srv2031724958=48, srv864947182=89, srv1759881130=41, srv2012728067=46, srv1213743231=9, srv332846821=66, srv1274246403=15, srv157444674=32, srv1076517605=2, srv513690103=73, srv1333747698=18, srv1368046539=20, srv240776984=56, srv203080753=47, srv601183459=80, srv2070069916=50, srv293429091=60, srv912098132=93, srv1571426524=31, srv279567172=59, srv316075896=64, srv551014491=76, srv254105722=58, srv1674017746=36, srv307270360=62, srv2060062430=49, srv619303459=81, srv898260838=92, srv1163646851=6, srv531443410=74, srv832411606=87, srv865771048=90, srv959189749=97, srv12538978=14} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-04 19:53:13,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-04 19:53:13,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-04 19:53:13,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-04 19:53:13,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-04 19:53:13,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-04 19:53:13,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-04 19:53:13,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-04 19:53:13,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-04 19:53:13,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-04 19:53:13,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-04 19:53:13,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-04 19:53:13,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-04 19:53:13,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-04 19:53:13,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-04 19:53:13,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-04 19:53:13,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-04 19:53:13,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-04 19:53:13,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-04 19:53:13,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-04 19:53:13,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-04 19:53:13,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-04 19:53:13,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-04 19:53:13,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-04 19:53:13,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-04 19:53:13,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-04 19:53:13,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-04 19:53:13,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-04 19:53:13,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-04 19:53:13,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-04 19:53:13,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-04 19:53:13,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-02-04 19:53:13,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-02-04 19:53:13,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-02-04 19:53:13,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-02-04 19:53:13,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-02-04 19:53:13,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-02-04 19:53:13,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-02-04 19:53:13,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-02-04 19:53:13,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-02-04 19:53:13,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-02-04 19:53:13,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-02-04 19:53:13,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-02-04 19:53:13,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-02-04 19:53:13,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-02-04 19:53:13,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-02-04 19:53:13,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-02-04 19:53:13,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-02-04 19:53:13,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-02-04 19:53:13,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-02-04 19:53:13,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-02-04 19:53:13,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-02-04 19:53:13,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-02-04 19:53:13,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-02-04 19:53:13,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-02-04 19:53:13,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-02-04 19:53:13,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-02-04 19:53:13,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-02-04 19:53:13,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-02-04 19:53:13,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-02-04 19:53:13,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-02-04 19:53:13,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-02-04 19:53:13,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-02-04 19:53:13,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-02-04 19:53:13,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-02-04 19:53:13,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-02-04 19:53:13,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-02-04 19:53:13,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-02-04 19:53:13,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-02-04 19:53:13,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-02-04 19:53:13,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-02-04 19:53:13,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-02-04 19:53:13,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-02-04 19:53:13,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-02-04 19:53:13,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-02-04 19:53:13,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-02-04 19:53:13,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-02-04 19:53:13,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-02-04 19:53:13,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-02-04 19:53:13,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-02-04 19:53:13,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-02-04 19:53:13,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-02-04 19:53:13,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-02-04 19:53:13,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-02-04 19:53:13,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-02-04 19:53:13,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-02-04 19:53:13,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-02-04 19:53:13,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-02-04 19:53:13,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-02-04 19:53:13,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-02-04 19:53:13,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-02-04 19:53:13,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-02-04 19:53:13,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-02-04 19:53:13,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-02-04 19:53:13,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-02-04 19:53:13,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-02-04 19:53:13,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-02-04 19:53:13,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-02-04 19:53:13,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-02-04 19:53:13,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-02-04 19:53:13,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-02-04 19:53:13,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-02-04 19:53:13,933 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=6.744069053831948E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014982219165849825); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-04 19:53:13,934 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1163646851:83 , srv1333747698:83 , srv1459004040:83 , srv1801406504:83 , srv252661560:83 , srv298516002:83 , srv354865752:83 , srv507429626:83 , srv934002937:83 , srv967831955:83 , srv1005456092:84 , srv1076517605:84 , srv1082237724:84 , srv1098953972:84 , srv1132094094:84 , srv1212279381:84 , srv1213113269:84 , srv1218555354:84 , srv1238272463:84 , srv124184945:84 , srv1249964249:84 , srv12538978:84 , srv1274246403:84 , srv1289903084:84 , srv1302196801:84 , srv1362646491:84 , srv1368046539:84 , srv1375511316:84 , srv13902331:84 , srv1399730236:84 , srv1405712787:84 , srv1442079598:84 , srv1519589761:84 , srv1568951186:84 , srv1570490020:84 , srv1571426524:84 , srv157444674:84 , srv1593532725:84 , srv1669416539:84 , srv1674017746:84 , srv1692617870:84 , srv1709026965:84 , srv1726539884:84 , srv1738760314:84 , srv1759881130:84 , srv1778355197:84 , srv180830820:84 , srv1975724219:84 , srv2031724958:84 , srv2060062430:84 , srv2070069916:84 , srv2076515261:84 , srv2086333527:84 , srv2101751006:84 , srv2110337982:84 , srv254105722:84 , srv293429091:84 , srv307270360:84 , srv313548538:84 , srv316075896:84 , srv332846821:84 , srv397372999:84 , srv432926130:84 , srv445412730:84 , srv513690103:84 , srv531443410:84 , srv547153974:84 , srv551014491:84 , srv570812977:84 , srv575862091:84 , srv580132628:84 , srv601183459:84 , srv619303459:84 , srv657098906:84 , srv672172881:84 , srv783108583:84 , srv795097099:84 , srv801830680:84 , srv832411606:84 , srv854340963:84 , srv864947182:84 , srv865771048:84 , srv88320294:84 , srv898260838:84 , srv912098132:84 , srv952943396:84 , srv953220300:84 , srv959189749:84 , srv960295165:84 , srv1032862408:85 , srv1213743231:85 , srv1537132568:85 , srv1588870531:85 , srv2012728067:85 , srv203080753:85 , srv2115844579:85 , srv240776984:85 , srv279567172:85 , srv317719923:85 , srv363520183:684 }
2024-02-04 19:53:13,961 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=336 (was 305) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=2202 (was 3150)
2024-02-04 19:53:13,968 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=336, ProcessCount=168, AvailableMemoryMB=2202
2024-02-04 19:53:13,969 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-02-04 19:53:13,969 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2024-02-04 19:53:13,989 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1004835918:84 , srv1017518623:84 , srv1040462797:84 , srv1121000669:84 , srv1142658090:84 , srv1167040725:84 , srv1281577829:84 , srv1411112385:84 , srv1475226765:84 , srv1700009715:84 , srv1714544859:84 , srv1788212133:84 , srv1812910659:84 , srv1874062688:84 , srv1901451137:84 , srv1907313777:84 , srv2049458719:84 , srv263726617:84 , srv270962449:84 , srv327264019:84 , srv477547596:84 , srv632052236:84 , srv64102068:84 , srv704856809:84 , srv746508635:84 , srv76594886:84 , srv79819072:84 , srv856439711:84 , srv920524441:84 , srv1904030592:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2024-02-04 19:53:13,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-04 19:53:13,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1874062688=13, srv1017518623=1, srv1812910659=12, srv1142658090=4, srv632052236=22, srv64102068=23, srv1040462797=2, srv1901451137=14, srv920524441=29, srv1004835918=0, srv1788212133=11, srv270962449=19, srv76594886=26, srv1121000669=3, srv1907313777=16, srv1281577829=6, srv746508635=25, srv704856809=24, srv1714544859=10, srv1700009715=9, srv1411112385=7, srv2049458719=17, srv1904030592=15, srv477547596=21, srv856439711=28, srv1475226765=8, srv263726617=18, srv79819072=27, srv1167040725=5, srv327264019=20} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-04 19:53:14,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-04 19:53:14,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-04 19:53:14,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-04 19:53:14,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-04 19:53:14,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-04 19:53:14,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-04 19:53:14,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-04 19:53:14,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-04 19:53:14,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-04 19:53:14,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-04 19:53:14,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-04 19:53:14,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-04 19:53:14,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-04 19:53:14,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-04 19:53:14,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-04 19:53:14,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-04 19:53:14,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-04 19:53:14,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-04 19:53:14,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-04 19:53:14,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-04 19:53:14,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-04 19:53:14,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-04 19:53:14,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-04 19:53:14,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-04 19:53:14,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-04 19:53:14,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-04 19:53:14,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-04 19:53:14,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-04 19:53:14,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-04 19:53:14,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-04 19:53:14,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-02-04 19:53:14,033 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1471006119004934, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-04 19:53:14,034 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912656552489101, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1471006119004934, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2024-02-04 19:55:14,013 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6380168 different iterations.  Found a solution that moves 1843 regions; Going from a computed imbalance of 0.9912656552489101 to a new imbalance of 4.3016539410932495E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6825925925925926, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-04 19:55:14,022 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1004835918:90 , srv1017518623:90 , srv1040462797:90 , srv1121000669:90 , srv1142658090:90 , srv1167040725:90 , srv1281577829:90 , srv1411112385:90 , srv1475226765:90 , srv1700009715:90 , srv1714544859:90 , srv1788212133:90 , srv1812910659:90 , srv1874062688:90 , srv1901451137:90 , srv1904030592:90 , srv1907313777:90 , srv2049458719:90 , srv263726617:90 , srv270962449:90 , srv327264019:90 , srv477547596:90 , srv632052236:90 , srv64102068:90 , srv704856809:90 , srv746508635:90 , srv76594886:90 , srv79819072:90 , srv856439711:90 , srv920524441:90 }
2024-02-04 19:55:14,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-04 19:55:14,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1874062688=13, srv1017518623=1, srv1812910659=12, srv1142658090=4, srv632052236=22, srv64102068=23, srv1040462797=2, srv1901451137=14, srv920524441=29, srv1004835918=0, srv1788212133=11, srv270962449=19, srv76594886=26, srv1121000669=3, srv1907313777=16, srv1281577829=6, srv746508635=25, srv704856809=24, srv1714544859=10, srv1700009715=9, srv1411112385=7, srv2049458719=17, srv1904030592=15, srv477547596=21, srv856439711=28, srv1475226765=8, srv263726617=18, srv79819072=27, srv1167040725=5, srv327264019=20} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-04 19:55:14,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-04 19:55:14,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-04 19:55:14,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-04 19:55:14,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-04 19:55:14,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-04 19:55:14,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-04 19:55:14,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-04 19:55:14,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-04 19:55:14,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-04 19:55:14,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-04 19:55:14,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-04 19:55:14,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-04 19:55:14,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-04 19:55:14,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-04 19:55:14,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-04 19:55:14,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-04 19:55:14,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-04 19:55:14,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-04 19:55:14,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-04 19:55:14,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-04 19:55:14,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-04 19:55:14,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-04 19:55:14,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-04 19:55:14,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-04 19:55:14,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-04 19:55:14,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-04 19:55:14,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-04 19:55:14,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-04 19:55:14,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-04 19:55:14,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-04 19:55:14,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-02-04 19:55:14,045 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-04 19:55:14,045 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1004835918:90 , srv1017518623:90 , srv1040462797:90 , srv1121000669:90 , srv1142658090:90 , srv1167040725:90 , srv1281577829:90 , srv1411112385:90 , srv1475226765:90 , srv1700009715:90 , srv1714544859:90 , srv1788212133:90 , srv1812910659:90 , srv1874062688:90 , srv1901451137:90 , srv1904030592:90 , srv1907313777:90 , srv2049458719:90 , srv263726617:90 , srv270962449:90 , srv327264019:90 , srv477547596:90 , srv632052236:90 , srv64102068:90 , srv704856809:90 , srv746508635:90 , srv76594886:90 , srv79819072:90 , srv856439711:90 , srv920524441:90 }
2024-02-04 19:55:14,062 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=281 (was 336), ProcessCount=167 (was 168), AvailableMemoryMB=2283 (was 2202) - AvailableMemoryMB LEAK? -