2023-03-10 17:54:19,058 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-03-10 17:54:19,375 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-03-10 17:54:19,493 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-03-10 17:54:19,560 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-10 17:54:19,567 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-03-10 17:54:19,579 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=296, ProcessCount=170, AvailableMemoryMB=6262
2023-03-10 17:54:19,581 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-10 17:54:19,581 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-03-10 17:54:19,713 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1004895421:84 , srv1018872618:84 , srv1020338849:84 , srv1024054168:84 , srv1038477241:84 , srv1064116179:84 , srv1073531708:84 , srv1133281146:84 , srv1169806331:84 , srv1196331402:84 , srv1253559202:84 , srv1278206452:84 , srv1284518950:84 , srv129862088:84 , srv1366165215:84 , srv1439793185:84 , srv1463596478:84 , srv1479532665:84 , srv1502142691:84 , srv1506122511:84 , srv1507807531:84 , srv1508419274:84 , srv1516828743:84 , srv153501890:84 , srv1540869358:84 , srv1542452481:84 , srv1549249205:84 , srv1550083453:84 , srv1568269093:84 , srv1574977860:84 , srv1580776970:84 , srv1620832867:84 , srv1641575347:84 , srv165799307:84 , srv1673465755:84 , srv1729448746:84 , srv1851796591:84 , srv1865008094:84 , srv1866757769:84 , srv186683283:84 , srv1919156950:84 , srv1920528261:84 , srv1921065908:84 , srv1938520253:84 , srv2001203183:84 , srv2038002829:84 , srv2049312436:84 , srv2084167122:84 , srv2111645194:84 , srv247960180:84 , srv249759404:84 , srv259038048:84 , srv269846841:84 , srv308632236:84 , srv355426541:84 , srv370177539:84 , srv382121337:84 , srv39147540:84 , srv392869112:84 , srv410806521:84 , srv427630011:84 , srv45520411:84 , srv455358626:84 , srv470282197:84 , srv47731334:84 , srv481507048:84 , srv48687583:84 , srv487215816:84 , srv506341889:84 , srv513238020:84 , srv515169093:84 , srv519327204:84 , srv555659488:84 , srv587171146:84 , srv628304812:84 , srv629968148:84 , srv630419271:84 , srv661097309:84 , srv67175675:84 , srv678675974:84 , srv698232176:84 , srv737490650:84 , srv747607546:84 , srv75202186:84 , srv765643683:84 , srv766610566:84 , srv77201928:84 , srv788462944:84 , srv789125354:84 , srv816117667:84 , srv849162527:84 , srv858025972:84 , srv862276012:84 , srv899715247:84 , srv899763228:84 , srv90296419:84 , srv924473209:84 , srv951897518:84 , srv971278430:84 , srv727574013:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-03-10 17:54:19,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-10 17:54:19,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv75202186=84, srv1550083453=27, srv1278206452=11, srv1580776970=30, srv816117667=90, srv1479532665=17, srv2111645194=48, srv630419271=76, srv1542452481=25, srv427630011=60, srv90296419=96, srv2001203183=44, srv1253559202=10, srv259038048=51, srv858025972=92, srv1169806331=8, srv1540869358=24, srv788462944=88, srv971278430=99, srv555659488=72, srv515169093=70, srv186683283=39, srv355426541=54, srv2038002829=45, srv899763228=95, srv1018872618=1, srv1507807531=20, srv45520411=61, srv1196331402=9, srv455358626=62, srv1133281146=7, srv1284518950=12, srv77201928=87, srv1064116179=5, srv1641575347=32, srv1620832867=31, srv470282197=63, srv481507048=65, srv513238020=69, srv1024054168=3, srv628304812=74, srv1938520253=43, srv1920528261=41, srv392869112=58, srv370177539=55, srv1439793185=15, srv849162527=91, srv899715247=94, srv1463596478=16, srv269846841=52, srv506341889=68, srv1502142691=18, srv629968148=75, srv765643683=85, srv1568269093=28, srv1673465755=34, srv153501890=23, srv766610566=86, srv789125354=89, srv487215816=67, srv862276012=93, srv249759404=50, srv1020338849=2, srv1921065908=42, srv1073531708=6, srv1549249205=26, srv129862088=13, srv1508419274=21, srv519327204=71, srv661097309=77, srv1729448746=35, srv747607546=83, srv587171146=73, srv698232176=80, srv410806521=59, srv165799307=33, srv308632236=53, srv1866757769=38, srv1865008094=37, srv1038477241=4, srv247960180=49, srv727574013=81, srv737490650=82, srv1574977860=29, srv1516828743=22, srv924473209=97, srv1851796591=36, srv2049312436=46, srv1919156950=40, srv1366165215=14, srv382121337=56, srv67175675=78, srv678675974=79, srv951897518=98, srv1004895421=0, srv2084167122=47, srv39147540=57, srv47731334=64, srv48687583=66, srv1506122511=19} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-10 17:54:19,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-10 17:54:19,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-10 17:54:19,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-10 17:54:19,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-10 17:54:19,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-10 17:54:19,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-10 17:54:19,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-10 17:54:19,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-10 17:54:19,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-10 17:54:19,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-10 17:54:19,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-10 17:54:19,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-10 17:54:19,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-10 17:54:19,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-10 17:54:19,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-10 17:54:19,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-10 17:54:19,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-10 17:54:19,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-10 17:54:19,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-10 17:54:19,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-10 17:54:19,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-10 17:54:19,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-10 17:54:19,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-10 17:54:19,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-10 17:54:19,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-10 17:54:19,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-10 17:54:19,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-10 17:54:19,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-10 17:54:19,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-10 17:54:19,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-10 17:54:19,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-10 17:54:19,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-10 17:54:19,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-10 17:54:19,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-10 17:54:19,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-10 17:54:19,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-10 17:54:19,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-10 17:54:19,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-10 17:54:19,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-10 17:54:19,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-10 17:54:19,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-10 17:54:19,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-10 17:54:19,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-10 17:54:19,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-10 17:54:19,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-10 17:54:19,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-10 17:54:19,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-10 17:54:19,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-10 17:54:19,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-10 17:54:19,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-10 17:54:19,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-10 17:54:19,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-10 17:54:19,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-10 17:54:19,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-10 17:54:19,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-10 17:54:19,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-10 17:54:19,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-10 17:54:19,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-10 17:54:19,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-10 17:54:19,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-10 17:54:19,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-10 17:54:19,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-10 17:54:19,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-10 17:54:19,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-10 17:54:19,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-10 17:54:19,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-10 17:54:19,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-10 17:54:19,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-10 17:54:19,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-10 17:54:19,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-10 17:54:19,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-10 17:54:19,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-10 17:54:19,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-10 17:54:19,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-10 17:54:19,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-10 17:54:19,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-10 17:54:19,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-10 17:54:19,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-10 17:54:19,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-10 17:54:19,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-10 17:54:19,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-10 17:54:19,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-10 17:54:19,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-10 17:54:19,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-10 17:54:19,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-10 17:54:19,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-10 17:54:19,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-10 17:54:19,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-10 17:54:19,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-10 17:54:19,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-10 17:54:19,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-10 17:54:19,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-10 17:54:19,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-10 17:54:19,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-10 17:54:19,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-10 17:54:19,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-10 17:54:19,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-10 17:54:19,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-10 17:54:19,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-10 17:54:19,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-10 17:54:19,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-10 17:54:20,028 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-10 17:54:20,028 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-10 17:54:20,029 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-10 17:54:20,379 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 521 ms to try 5000 different iterations.  Found a solution that moves 4680 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1815161232355938. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039371227157294776, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.056792294888462574, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.52, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039371227157294776, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17658333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24508333333333332, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-10 17:54:20,390 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv427630011:82 , srv629968148:82 , srv1550083453:83 , srv1729448746:83 , srv1938520253:83 , srv515169093:83 , srv77201928:83 , srv789125354:83 , srv971278430:83 , srv1038477241:84 , srv129862088:84 , srv1479532665:84 , srv1502142691:84 , srv1549249205:84 , srv1673465755:84 , srv1865008094:84 , srv2084167122:84 , srv410806521:84 , srv487215816:84 , srv849162527:84 , srv858025972:84 , srv899715247:84 , srv1004895421:85 , srv1253559202:85 , srv1540869358:85 , srv1542452481:85 , srv165799307:85 , srv1921065908:85 , srv259038048:85 , srv45520411:85 , srv470282197:85 , srv47731334:85 , srv737490650:85 , srv747607546:85 , srv766610566:85 , srv788462944:85 , srv924473209:85 , srv1018872618:86 , srv1366165215:86 , srv1439793185:86 , srv1506122511:86 , srv1507807531:86 , srv1516828743:86 , srv1866757769:86 , srv2049312436:86 , srv249759404:86 , srv355426541:86 , srv382121337:86 , srv506341889:86 , srv678675974:86 , srv75202186:86 , srv862276012:86 , srv951897518:86 , srv1020338849:87 , srv1024054168:87 , srv1278206452:87 , srv1284518950:87 , srv153501890:87 , srv1568269093:87 , srv1620832867:87 , srv247960180:87 , srv269846841:87 , srv308632236:87 , srv370177539:87 , srv39147540:87 , srv455358626:87 , srv555659488:87 , srv587171146:87 , srv628304812:87 , srv67175675:87 , srv90296419:87 , srv1508419274:88 , srv1641575347:88 , srv1851796591:88 , srv186683283:88 , srv1919156950:88 , srv2001203183:88 , srv48687583:88 , srv519327204:88 , srv1463596478:89 , srv1574977860:89 , srv1580776970:89 , srv2111645194:89 , srv630419271:89 , srv698232176:89 , srv816117667:89 , srv1073531708:90 , srv1196331402:90 , srv1920528261:90 , srv2038002829:90 , srv392869112:90 , srv481507048:90 , srv513238020:90 , srv661097309:90 , srv899763228:90 , srv1064116179:91 , srv1169806331:91 , srv1133281146:92 , srv765643683:95 , srv727574013:440 }
2023-03-10 17:54:20,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-10 17:54:20,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv75202186=84, srv1550083453=27, srv1278206452=11, srv1580776970=30, srv816117667=90, srv1479532665=17, srv2111645194=48, srv630419271=76, srv1542452481=25, srv427630011=60, srv90296419=96, srv2001203183=44, srv1253559202=10, srv259038048=51, srv858025972=92, srv1169806331=8, srv1540869358=24, srv788462944=88, srv971278430=99, srv555659488=72, srv515169093=70, srv186683283=39, srv355426541=54, srv2038002829=45, srv899763228=95, srv1018872618=1, srv1507807531=20, srv45520411=61, srv1196331402=9, srv455358626=62, srv1133281146=7, srv1284518950=12, srv77201928=87, srv1064116179=5, srv1641575347=32, srv1620832867=31, srv470282197=63, srv481507048=65, srv513238020=69, srv1024054168=3, srv628304812=74, srv1938520253=43, srv1920528261=41, srv392869112=58, srv370177539=55, srv1439793185=15, srv849162527=91, srv899715247=94, srv1463596478=16, srv269846841=52, srv506341889=68, srv1502142691=18, srv629968148=75, srv765643683=85, srv1568269093=28, srv1673465755=34, srv153501890=23, srv766610566=86, srv789125354=89, srv487215816=67, srv862276012=93, srv249759404=50, srv1020338849=2, srv1921065908=42, srv1073531708=6, srv1549249205=26, srv129862088=13, srv1508419274=21, srv519327204=71, srv661097309=77, srv1729448746=35, srv747607546=83, srv587171146=73, srv698232176=80, srv410806521=59, srv165799307=33, srv308632236=53, srv1866757769=38, srv1865008094=37, srv1038477241=4, srv247960180=49, srv727574013=81, srv737490650=82, srv1574977860=29, srv1516828743=22, srv924473209=97, srv1851796591=36, srv2049312436=46, srv1919156950=40, srv1366165215=14, srv382121337=56, srv67175675=78, srv678675974=79, srv951897518=98, srv1004895421=0, srv2084167122=47, srv39147540=57, srv47731334=64, srv48687583=66, srv1506122511=19} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-10 17:54:20,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-10 17:54:20,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-10 17:54:20,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-10 17:54:20,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-10 17:54:20,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-10 17:54:20,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-10 17:54:20,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-10 17:54:20,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-10 17:54:20,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-10 17:54:20,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-10 17:54:20,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-10 17:54:20,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-10 17:54:20,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-10 17:54:20,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-10 17:54:20,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-10 17:54:20,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-10 17:54:20,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-10 17:54:20,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-10 17:54:20,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-10 17:54:20,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-10 17:54:20,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-10 17:54:20,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-10 17:54:20,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-10 17:54:20,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-10 17:54:20,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-10 17:54:20,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-10 17:54:20,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-10 17:54:20,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-10 17:54:20,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-10 17:54:20,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-10 17:54:20,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-10 17:54:20,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-10 17:54:20,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-10 17:54:20,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-10 17:54:20,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-10 17:54:20,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-10 17:54:20,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-10 17:54:20,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-10 17:54:20,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-10 17:54:20,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-10 17:54:20,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-10 17:54:20,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-10 17:54:20,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-10 17:54:20,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-10 17:54:20,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-10 17:54:20,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-10 17:54:20,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-10 17:54:20,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-10 17:54:20,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-10 17:54:20,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-10 17:54:20,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-10 17:54:20,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-10 17:54:20,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-10 17:54:20,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-10 17:54:20,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-10 17:54:20,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-10 17:54:20,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-10 17:54:20,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-10 17:54:20,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-10 17:54:20,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-10 17:54:20,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-10 17:54:20,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-10 17:54:20,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-10 17:54:20,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-10 17:54:20,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-10 17:54:20,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-10 17:54:20,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-10 17:54:20,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-10 17:54:20,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-10 17:54:20,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-10 17:54:20,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-10 17:54:20,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-10 17:54:20,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-10 17:54:20,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-10 17:54:20,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-10 17:54:20,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-10 17:54:20,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-10 17:54:20,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-10 17:54:20,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-10 17:54:20,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-10 17:54:20,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-10 17:54:20,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-10 17:54:20,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-10 17:54:20,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-10 17:54:20,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-10 17:54:20,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-10 17:54:20,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-10 17:54:20,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-10 17:54:20,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-10 17:54:20,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-10 17:54:20,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-10 17:54:20,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-10 17:54:20,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-10 17:54:20,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-10 17:54:20,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-10 17:54:20,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-10 17:54:20,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-10 17:54:20,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-10 17:54:20,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-10 17:54:20,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-10 17:54:20,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-10 17:54:20,589 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039371227157294776, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.056792294888462574, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039371227157294776, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17658333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24508333333333332, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-10 17:54:20,590 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-10 17:54:20,590 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18148335317518524, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039371227157294776, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.056792294888462574, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039371227157294776, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17658333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24508333333333332, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-10 17:54:20,901 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 461 ms to try 5000 different iterations.  Found a solution that moves 2601 regions; Going from a computed imbalance of 0.18148335317518524 to a new imbalance of 0.015548280265688117. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0220656662630968); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.036456391242164876, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.289, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0220656662630968); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012416666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04533333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-10 17:54:20,914 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv765643683:77 , srv1196331402:79 , srv1133281146:80 , srv1169806331:80 , srv1463596478:81 , srv481507048:81 , srv628304812:81 , srv1073531708:82 , srv1278206452:82 , srv1580776970:82 , srv1920528261:82 , srv392869112:82 , srv630419271:82 , srv1284518950:83 , srv1620832867:83 , srv2038002829:83 , srv2111645194:83 , srv269846841:83 , srv506341889:83 , srv661097309:83 , srv899763228:83 , srv1064116179:84 , srv1508419274:84 , srv1540869358:84 , srv1574977860:84 , srv1641575347:84 , srv186683283:84 , srv1919156950:84 , srv247960180:84 , srv249759404:84 , srv67175675:84 , srv816117667:84 , srv1024054168:85 , srv1366165215:85 , srv153501890:85 , srv1866757769:85 , srv259038048:85 , srv455358626:85 , srv47731334:85 , srv513238020:85 , srv555659488:85 , srv678675974:85 , srv698232176:85 , srv766610566:85 , srv1004895421:86 , srv1253559202:86 , srv1439793185:86 , srv1507807531:86 , srv1516828743:86 , srv1568269093:86 , srv1851796591:86 , srv1921065908:86 , srv2049312436:86 , srv370177539:86 , srv45520411:86 , srv48687583:86 , srv75202186:86 , srv858025972:86 , srv90296419:86 , srv951897518:86 , srv1018872618:87 , srv1020338849:87 , srv1479532665:87 , srv1542452481:87 , srv165799307:87 , srv1673465755:87 , srv2001203183:87 , srv308632236:87 , srv39147540:87 , srv427630011:87 , srv470282197:87 , srv587171146:87 , srv737490650:87 , srv77201928:87 , srv862276012:87 , srv899715247:87 , srv924473209:87 , srv1038477241:88 , srv129862088:88 , srv1506122511:88 , srv1550083453:88 , srv1865008094:88 , srv2084167122:88 , srv382121337:88 , srv410806521:88 , srv519327204:88 , srv747607546:88 , srv788462944:88 , srv1549249205:89 , srv355426541:89 , srv515169093:89 , srv1729448746:90 , srv1938520253:90 , srv487215816:90 , srv629968148:90 , srv789125354:90 , srv849162527:90 , srv971278430:90 , srv1502142691:91 , srv727574013:530 }
2023-03-10 17:54:20,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-10 17:54:20,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv75202186=84, srv1550083453=27, srv1278206452=11, srv1580776970=30, srv816117667=90, srv1479532665=17, srv2111645194=48, srv630419271=76, srv1542452481=25, srv427630011=60, srv90296419=96, srv2001203183=44, srv1253559202=10, srv259038048=51, srv858025972=92, srv1169806331=8, srv1540869358=24, srv788462944=88, srv971278430=99, srv555659488=72, srv515169093=70, srv186683283=39, srv355426541=54, srv2038002829=45, srv899763228=95, srv1018872618=1, srv1507807531=20, srv45520411=61, srv1196331402=9, srv455358626=62, srv1133281146=7, srv1284518950=12, srv77201928=87, srv1064116179=5, srv1641575347=32, srv1620832867=31, srv470282197=63, srv481507048=65, srv513238020=69, srv1024054168=3, srv628304812=74, srv1938520253=43, srv1920528261=41, srv392869112=58, srv370177539=55, srv1439793185=15, srv849162527=91, srv899715247=94, srv1463596478=16, srv269846841=52, srv506341889=68, srv1502142691=18, srv629968148=75, srv765643683=85, srv1568269093=28, srv1673465755=34, srv153501890=23, srv766610566=86, srv789125354=89, srv487215816=67, srv862276012=93, srv249759404=50, srv1020338849=2, srv1921065908=42, srv1073531708=6, srv1549249205=26, srv129862088=13, srv1508419274=21, srv519327204=71, srv661097309=77, srv1729448746=35, srv747607546=83, srv587171146=73, srv698232176=80, srv410806521=59, srv165799307=33, srv308632236=53, srv1866757769=38, srv1865008094=37, srv1038477241=4, srv247960180=49, srv727574013=81, srv737490650=82, srv1574977860=29, srv1516828743=22, srv924473209=97, srv1851796591=36, srv2049312436=46, srv1919156950=40, srv1366165215=14, srv382121337=56, srv67175675=78, srv678675974=79, srv951897518=98, srv1004895421=0, srv2084167122=47, srv39147540=57, srv47731334=64, srv48687583=66, srv1506122511=19} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-10 17:54:20,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-10 17:54:20,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-10 17:54:20,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-10 17:54:20,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-10 17:54:20,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-10 17:54:20,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-10 17:54:20,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-10 17:54:20,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-10 17:54:20,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-10 17:54:20,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-10 17:54:20,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-10 17:54:20,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-10 17:54:20,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-10 17:54:20,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-10 17:54:20,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-10 17:54:20,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-10 17:54:20,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-10 17:54:20,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-10 17:54:20,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-10 17:54:20,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-10 17:54:20,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-10 17:54:20,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-10 17:54:20,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-10 17:54:20,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-10 17:54:20,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-10 17:54:20,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-10 17:54:20,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-10 17:54:20,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-10 17:54:20,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-10 17:54:20,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-10 17:54:20,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-10 17:54:20,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-10 17:54:20,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-10 17:54:20,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-10 17:54:20,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-10 17:54:20,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-10 17:54:20,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-10 17:54:20,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-10 17:54:20,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-10 17:54:20,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-10 17:54:20,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-10 17:54:20,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-10 17:54:20,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-10 17:54:20,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-10 17:54:20,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-10 17:54:20,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-10 17:54:20,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-10 17:54:20,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-10 17:54:20,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-10 17:54:20,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-10 17:54:20,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-10 17:54:20,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-10 17:54:20,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-10 17:54:20,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-10 17:54:20,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-10 17:54:20,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-10 17:54:20,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-10 17:54:20,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-10 17:54:20,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-10 17:54:20,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-10 17:54:20,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-10 17:54:20,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-10 17:54:20,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-10 17:54:20,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-10 17:54:20,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-10 17:54:20,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-10 17:54:20,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-10 17:54:20,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-10 17:54:20,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-10 17:54:20,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-10 17:54:20,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-10 17:54:20,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-10 17:54:20,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-10 17:54:20,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-10 17:54:20,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-10 17:54:20,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-10 17:54:20,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-10 17:54:20,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-10 17:54:20,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-10 17:54:20,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-10 17:54:20,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-10 17:54:20,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-10 17:54:20,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-10 17:54:20,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-10 17:54:20,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-10 17:54:20,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-10 17:54:20,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-10 17:54:20,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-10 17:54:20,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-10 17:54:20,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-10 17:54:20,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-10 17:54:20,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-10 17:54:20,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-10 17:54:20,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-10 17:54:20,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-10 17:54:20,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-10 17:54:20,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-10 17:54:20,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-10 17:54:20,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-10 17:54:20,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-10 17:54:20,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-10 17:54:21,169 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0220656662630968); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.036456391242164876, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0220656662630968); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012416666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04533333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-10 17:54:21,170 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-10 17:54:21,170 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.015530067674422601, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0220656662630968); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.036456391242164876, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0220656662630968); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012416666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04533333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-10 17:54:21,513 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 549 ms to try 5000 different iterations.  Found a solution that moves 967 regions; Going from a computed imbalance of 0.015530067674422601 to a new imbalance of 7.432469731544654E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0066647958467430145); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009813067629253163); MoveCostFunction : (multiplier=7.0, imbalance=0.10744444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0066647958467430145); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-10 17:54:21,519 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1502142691:82 , srv2001203183:82 , srv513238020:82 , srv519327204:82 , srv1064116179:83 , srv355426541:83 , srv39147540:83 , srv587171146:83 , srv661097309:83 , srv698232176:83 , srv1018872618:84 , srv1020338849:84 , srv1038477241:84 , srv1439793185:84 , srv1506122511:84 , srv1574977860:84 , srv1729448746:84 , srv1851796591:84 , srv186683283:84 , srv1938520253:84 , srv2038002829:84 , srv2084167122:84 , srv308632236:84 , srv382121337:84 , srv48687583:84 , srv487215816:84 , srv747607546:84 , srv849162527:84 , srv1073531708:85 , srv1133281146:85 , srv1507807531:85 , srv1508419274:85 , srv1516828743:85 , srv153501890:85 , srv1542452481:85 , srv1549249205:85 , srv1568269093:85 , srv1641575347:85 , srv1919156950:85 , srv1920528261:85 , srv392869112:85 , srv455358626:85 , srv470282197:85 , srv515169093:85 , srv555659488:85 , srv630419271:85 , srv788462944:85 , srv789125354:85 , srv816117667:85 , srv862276012:85 , srv899763228:85 , srv90296419:85 , srv924473209:85 , srv951897518:85 , srv971278430:85 , srv1004895421:86 , srv1024054168:86 , srv1169806331:86 , srv1253559202:86 , srv129862088:86 , srv1366165215:86 , srv1479532665:86 , srv165799307:86 , srv1673465755:86 , srv1865008094:86 , srv1866757769:86 , srv1921065908:86 , srv2111645194:86 , srv247960180:86 , srv370177539:86 , srv410806521:86 , srv481507048:86 , srv629968148:86 , srv67175675:86 , srv678675974:86 , srv737490650:86 , srv75202186:86 , srv765643683:86 , srv899715247:86 , srv1284518950:87 , srv1463596478:87 , srv1550083453:87 , srv1580776970:87 , srv1620832867:87 , srv2049312436:87 , srv259038048:87 , srv269846841:87 , srv45520411:87 , srv47731334:87 , srv77201928:87 , srv1540869358:88 , srv249759404:88 , srv766610566:88 , srv858025972:88 , srv1196331402:89 , srv427630011:89 , srv506341889:89 , srv628304812:89 , srv1278206452:91 , srv727574013:547 }
2023-03-10 17:54:21,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-10 17:54:21,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv75202186=84, srv1550083453=27, srv1278206452=11, srv1580776970=30, srv816117667=90, srv1479532665=17, srv2111645194=48, srv630419271=76, srv1542452481=25, srv427630011=60, srv90296419=96, srv2001203183=44, srv1253559202=10, srv259038048=51, srv858025972=92, srv1169806331=8, srv1540869358=24, srv788462944=88, srv971278430=99, srv555659488=72, srv515169093=70, srv186683283=39, srv355426541=54, srv2038002829=45, srv899763228=95, srv1018872618=1, srv1507807531=20, srv45520411=61, srv1196331402=9, srv455358626=62, srv1133281146=7, srv1284518950=12, srv77201928=87, srv1064116179=5, srv1641575347=32, srv1620832867=31, srv470282197=63, srv481507048=65, srv513238020=69, srv1024054168=3, srv628304812=74, srv1938520253=43, srv1920528261=41, srv392869112=58, srv370177539=55, srv1439793185=15, srv849162527=91, srv899715247=94, srv1463596478=16, srv269846841=52, srv506341889=68, srv1502142691=18, srv629968148=75, srv765643683=85, srv1568269093=28, srv1673465755=34, srv153501890=23, srv766610566=86, srv789125354=89, srv487215816=67, srv862276012=93, srv249759404=50, srv1020338849=2, srv1921065908=42, srv1073531708=6, srv1549249205=26, srv129862088=13, srv1508419274=21, srv519327204=71, srv661097309=77, srv1729448746=35, srv747607546=83, srv587171146=73, srv698232176=80, srv410806521=59, srv165799307=33, srv308632236=53, srv1866757769=38, srv1865008094=37, srv1038477241=4, srv247960180=49, srv727574013=81, srv737490650=82, srv1574977860=29, srv1516828743=22, srv924473209=97, srv1851796591=36, srv2049312436=46, srv1919156950=40, srv1366165215=14, srv382121337=56, srv67175675=78, srv678675974=79, srv951897518=98, srv1004895421=0, srv2084167122=47, srv39147540=57, srv47731334=64, srv48687583=66, srv1506122511=19} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-10 17:54:21,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-10 17:54:21,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-10 17:54:21,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-10 17:54:21,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-10 17:54:21,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-10 17:54:21,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-10 17:54:21,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-10 17:54:21,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-10 17:54:21,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-10 17:54:21,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-10 17:54:21,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-10 17:54:21,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-10 17:54:21,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-10 17:54:21,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-10 17:54:21,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-10 17:54:21,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-10 17:54:21,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-10 17:54:21,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-10 17:54:21,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-10 17:54:21,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-10 17:54:21,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-10 17:54:21,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-10 17:54:21,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-10 17:54:21,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-10 17:54:21,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-10 17:54:21,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-10 17:54:21,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-10 17:54:21,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-10 17:54:21,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-10 17:54:21,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-10 17:54:21,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-10 17:54:21,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-10 17:54:21,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-10 17:54:21,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-10 17:54:21,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-10 17:54:21,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-10 17:54:21,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-10 17:54:21,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-10 17:54:21,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-10 17:54:21,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-10 17:54:21,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-10 17:54:21,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-10 17:54:21,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-10 17:54:21,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-10 17:54:21,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-10 17:54:21,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-10 17:54:21,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-10 17:54:21,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-10 17:54:21,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-10 17:54:21,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-10 17:54:21,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-10 17:54:21,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-10 17:54:21,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-10 17:54:21,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-10 17:54:21,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-10 17:54:21,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-10 17:54:21,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-10 17:54:21,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-10 17:54:21,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-10 17:54:21,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-10 17:54:21,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-10 17:54:21,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-10 17:54:21,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-10 17:54:21,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-10 17:54:21,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-10 17:54:21,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-10 17:54:21,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-10 17:54:21,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-10 17:54:21,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-10 17:54:21,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-10 17:54:21,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-10 17:54:21,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-10 17:54:21,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-10 17:54:21,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-10 17:54:21,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-10 17:54:21,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-10 17:54:21,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-10 17:54:21,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-10 17:54:21,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-10 17:54:21,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-10 17:54:21,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-10 17:54:21,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-10 17:54:21,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-10 17:54:21,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-10 17:54:21,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-10 17:54:21,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-10 17:54:21,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-10 17:54:21,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-10 17:54:21,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-10 17:54:21,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-10 17:54:21,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-10 17:54:21,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-10 17:54:21,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-10 17:54:21,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-10 17:54:21,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-10 17:54:21,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-10 17:54:21,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-10 17:54:21,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-10 17:54:21,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-10 17:54:21,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-10 17:54:21,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-10 17:54:21,657 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0066647958467430145); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009813067629253163); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0066647958467430145); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-10 17:54:21,657 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-10 17:54:21,658 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=7.364758944332978E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0066647958467430145); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009813067629253163); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0066647958467430145); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-10 17:54:21,982 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 421 ms to try 5000 different iterations.  Found a solution that moves 255 regions; Going from a computed imbalance of 7.364758944332978E-4 to a new imbalance of 2.092413883853334E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.868392354571063E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0021711298242630987); MoveCostFunction : (multiplier=7.0, imbalance=0.028333333333333332, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.868392354571063E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-10 17:54:21,983 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1278206452:83 , srv2049312436:83 , srv370177539:83 , srv1064116179:84 , srv1196331402:84 , srv129862088:84 , srv1506122511:84 , srv1540869358:84 , srv1549249205:84 , srv1550083453:84 , srv1568269093:84 , srv165799307:84 , srv1851796591:84 , srv1865008094:84 , srv2038002829:84 , srv2111645194:84 , srv249759404:84 , srv308632236:84 , srv355426541:84 , srv382121337:84 , srv410806521:84 , srv427630011:84 , srv45520411:84 , srv481507048:84 , srv48687583:84 , srv487215816:84 , srv506341889:84 , srv519327204:84 , srv587171146:84 , srv629968148:84 , srv737490650:84 , srv75202186:84 , srv765643683:84 , srv766610566:84 , srv788462944:84 , srv789125354:84 , srv816117667:84 , srv849162527:84 , srv858025972:84 , srv862276012:84 , srv899763228:84 , srv971278430:84 , srv1004895421:85 , srv1018872618:85 , srv1020338849:85 , srv1024054168:85 , srv1073531708:85 , srv1133281146:85 , srv1169806331:85 , srv1253559202:85 , srv1284518950:85 , srv1366165215:85 , srv1463596478:85 , srv1479532665:85 , srv1502142691:85 , srv1507807531:85 , srv1508419274:85 , srv1516828743:85 , srv153501890:85 , srv1542452481:85 , srv1574977860:85 , srv1580776970:85 , srv1620832867:85 , srv1641575347:85 , srv1673465755:85 , srv1729448746:85 , srv1866757769:85 , srv1919156950:85 , srv1920528261:85 , srv1921065908:85 , srv1938520253:85 , srv2001203183:85 , srv247960180:85 , srv259038048:85 , srv269846841:85 , srv39147540:85 , srv392869112:85 , srv455358626:85 , srv470282197:85 , srv47731334:85 , srv513238020:85 , srv515169093:85 , srv555659488:85 , srv628304812:85 , srv630419271:85 , srv661097309:85 , srv67175675:85 , srv678675974:85 , srv698232176:85 , srv747607546:85 , srv77201928:85 , srv899715247:85 , srv90296419:85 , srv924473209:85 , srv951897518:85 , srv1439793185:86 , srv186683283:86 , srv2084167122:86 , srv1038477241:87 , srv727574013:625 }
2023-03-10 17:54:21,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-10 17:54:21,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv75202186=84, srv1550083453=27, srv1278206452=11, srv1580776970=30, srv816117667=90, srv1479532665=17, srv2111645194=48, srv630419271=76, srv1542452481=25, srv427630011=60, srv90296419=96, srv2001203183=44, srv1253559202=10, srv259038048=51, srv858025972=92, srv1169806331=8, srv1540869358=24, srv788462944=88, srv971278430=99, srv555659488=72, srv515169093=70, srv186683283=39, srv355426541=54, srv2038002829=45, srv899763228=95, srv1018872618=1, srv1507807531=20, srv45520411=61, srv1196331402=9, srv455358626=62, srv1133281146=7, srv1284518950=12, srv77201928=87, srv1064116179=5, srv1641575347=32, srv1620832867=31, srv470282197=63, srv481507048=65, srv513238020=69, srv1024054168=3, srv628304812=74, srv1938520253=43, srv1920528261=41, srv392869112=58, srv370177539=55, srv1439793185=15, srv849162527=91, srv899715247=94, srv1463596478=16, srv269846841=52, srv506341889=68, srv1502142691=18, srv629968148=75, srv765643683=85, srv1568269093=28, srv1673465755=34, srv153501890=23, srv766610566=86, srv789125354=89, srv487215816=67, srv862276012=93, srv249759404=50, srv1020338849=2, srv1921065908=42, srv1073531708=6, srv1549249205=26, srv129862088=13, srv1508419274=21, srv519327204=71, srv661097309=77, srv1729448746=35, srv747607546=83, srv587171146=73, srv698232176=80, srv410806521=59, srv165799307=33, srv308632236=53, srv1866757769=38, srv1865008094=37, srv1038477241=4, srv247960180=49, srv727574013=81, srv737490650=82, srv1574977860=29, srv1516828743=22, srv924473209=97, srv1851796591=36, srv2049312436=46, srv1919156950=40, srv1366165215=14, srv382121337=56, srv67175675=78, srv678675974=79, srv951897518=98, srv1004895421=0, srv2084167122=47, srv39147540=57, srv47731334=64, srv48687583=66, srv1506122511=19} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-10 17:54:22,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-10 17:54:22,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-10 17:54:22,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-10 17:54:22,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-10 17:54:22,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-10 17:54:22,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-10 17:54:22,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-10 17:54:22,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-10 17:54:22,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-10 17:54:22,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-10 17:54:22,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-10 17:54:22,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-10 17:54:22,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-10 17:54:22,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-10 17:54:22,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-10 17:54:22,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-10 17:54:22,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-10 17:54:22,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-10 17:54:22,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-10 17:54:22,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-10 17:54:22,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-10 17:54:22,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-10 17:54:22,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-10 17:54:22,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-10 17:54:22,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-10 17:54:22,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-10 17:54:22,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-10 17:54:22,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-10 17:54:22,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-10 17:54:22,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-10 17:54:22,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-10 17:54:22,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-10 17:54:22,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-10 17:54:22,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-10 17:54:22,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-10 17:54:22,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-10 17:54:22,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-10 17:54:22,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-10 17:54:22,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-10 17:54:22,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-10 17:54:22,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-10 17:54:22,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-10 17:54:22,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-10 17:54:22,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-10 17:54:22,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-10 17:54:22,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-10 17:54:22,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-10 17:54:22,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-10 17:54:22,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-10 17:54:22,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-10 17:54:22,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-10 17:54:22,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-10 17:54:22,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-10 17:54:22,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-10 17:54:22,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-10 17:54:22,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-10 17:54:22,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-10 17:54:22,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-10 17:54:22,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-10 17:54:22,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-10 17:54:22,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-10 17:54:22,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-10 17:54:22,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-10 17:54:22,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-10 17:54:22,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-10 17:54:22,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-10 17:54:22,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-10 17:54:22,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-10 17:54:22,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-10 17:54:22,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-10 17:54:22,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-10 17:54:22,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-10 17:54:22,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-10 17:54:22,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-10 17:54:22,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-10 17:54:22,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-10 17:54:22,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-10 17:54:22,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-10 17:54:22,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-10 17:54:22,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-10 17:54:22,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-10 17:54:22,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-10 17:54:22,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-10 17:54:22,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-10 17:54:22,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-10 17:54:22,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-10 17:54:22,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-10 17:54:22,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-10 17:54:22,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-10 17:54:22,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-10 17:54:22,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-10 17:54:22,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-10 17:54:22,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-10 17:54:22,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-10 17:54:22,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-10 17:54:22,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-10 17:54:22,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-10 17:54:22,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-10 17:54:22,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-10 17:54:22,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-10 17:54:22,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-10 17:54:22,133 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.913859067524721E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.868392354571063E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0021711298242630987); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.868392354571063E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-10 17:54:22,134 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1278206452:83 , srv2049312436:83 , srv370177539:83 , srv1064116179:84 , srv1196331402:84 , srv129862088:84 , srv1506122511:84 , srv1540869358:84 , srv1549249205:84 , srv1550083453:84 , srv1568269093:84 , srv165799307:84 , srv1851796591:84 , srv1865008094:84 , srv2038002829:84 , srv2111645194:84 , srv249759404:84 , srv308632236:84 , srv355426541:84 , srv382121337:84 , srv410806521:84 , srv427630011:84 , srv45520411:84 , srv481507048:84 , srv48687583:84 , srv487215816:84 , srv506341889:84 , srv519327204:84 , srv587171146:84 , srv629968148:84 , srv737490650:84 , srv75202186:84 , srv765643683:84 , srv766610566:84 , srv788462944:84 , srv789125354:84 , srv816117667:84 , srv849162527:84 , srv858025972:84 , srv862276012:84 , srv899763228:84 , srv971278430:84 , srv1004895421:85 , srv1018872618:85 , srv1020338849:85 , srv1024054168:85 , srv1073531708:85 , srv1133281146:85 , srv1169806331:85 , srv1253559202:85 , srv1284518950:85 , srv1366165215:85 , srv1463596478:85 , srv1479532665:85 , srv1502142691:85 , srv1507807531:85 , srv1508419274:85 , srv1516828743:85 , srv153501890:85 , srv1542452481:85 , srv1574977860:85 , srv1580776970:85 , srv1620832867:85 , srv1641575347:85 , srv1673465755:85 , srv1729448746:85 , srv1866757769:85 , srv1919156950:85 , srv1920528261:85 , srv1921065908:85 , srv1938520253:85 , srv2001203183:85 , srv247960180:85 , srv259038048:85 , srv269846841:85 , srv39147540:85 , srv392869112:85 , srv455358626:85 , srv470282197:85 , srv47731334:85 , srv513238020:85 , srv515169093:85 , srv555659488:85 , srv628304812:85 , srv630419271:85 , srv661097309:85 , srv67175675:85 , srv678675974:85 , srv698232176:85 , srv747607546:85 , srv77201928:85 , srv899715247:85 , srv90296419:85 , srv924473209:85 , srv951897518:85 , srv1439793185:86 , srv186683283:86 , srv2084167122:86 , srv1038477241:87 , srv727574013:625 }
2023-03-10 17:54:22,162 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=328 (was 296) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=5707 (was 6262)
2023-03-10 17:54:22,169 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=328, ProcessCount=170, AvailableMemoryMB=5706
2023-03-10 17:54:22,170 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-10 17:54:22,170 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-03-10 17:54:22,179 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1056402053:84 , srv1088126683:84 , srv1282662896:84 , srv1302632651:84 , srv132677575:84 , srv134068499:84 , srv1478690897:84 , srv1631180376:84 , srv1637442471:84 , srv1648081851:84 , srv1689451363:84 , srv1790317296:84 , srv1990019523:84 , srv2032415375:84 , srv2086376601:84 , srv2125125674:84 , srv235750987:84 , srv258342316:84 , srv322308056:84 , srv426469882:84 , srv464459456:84 , srv474506454:84 , srv54079576:84 , srv578911871:84 , srv668982758:84 , srv714320866:84 , srv794754819:84 , srv795449561:84 , srv798785744:84 , srv1464509088:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-03-10 17:54:22,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-10 17:54:22,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1464509088=6, srv132677575=4, srv714320866=26, srv1282662896=2, srv474506454=22, srv322308056=19, srv1790317296=12, srv1478690897=7, srv2086376601=15, srv668982758=25, srv1631180376=8, srv258342316=18, srv235750987=17, srv1689451363=11, srv795449561=28, srv1056402053=0, srv2032415375=14, srv1648081851=10, srv1990019523=13, srv1088126683=1, srv134068499=5, srv578911871=24, srv54079576=23, srv464459456=21, srv794754819=27, srv1302632651=3, srv2125125674=16, srv798785744=29, srv426469882=20, srv1637442471=9} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-10 17:54:22,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-10 17:54:22,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-10 17:54:22,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-10 17:54:22,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-10 17:54:22,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-10 17:54:22,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-10 17:54:22,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-10 17:54:22,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-10 17:54:22,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-10 17:54:22,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-10 17:54:22,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-10 17:54:22,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-10 17:54:22,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-10 17:54:22,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-10 17:54:22,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-10 17:54:22,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-10 17:54:22,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-10 17:54:22,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-10 17:54:22,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-10 17:54:22,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-10 17:54:22,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-10 17:54:22,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-10 17:54:22,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-10 17:54:22,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-10 17:54:22,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-10 17:54:22,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-10 17:54:22,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-10 17:54:22,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-10 17:54:22,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-10 17:54:22,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-10 17:54:22,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-03-10 17:54:22,205 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1195426025459478, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-10 17:54:22,206 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912569718101476, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1195426025459478, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-03-10 17:56:22,193 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5636885 different iterations.  Found a solution that moves 1861 regions; Going from a computed imbalance of 0.9912569718101476 to a new imbalance of 4.3436668390529226E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6892592592592592, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-10 17:56:22,198 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1056402053:90 , srv1088126683:90 , srv1282662896:90 , srv1302632651:90 , srv132677575:90 , srv134068499:90 , srv1464509088:90 , srv1478690897:90 , srv1631180376:90 , srv1637442471:90 , srv1648081851:90 , srv1689451363:90 , srv1790317296:90 , srv1990019523:90 , srv2032415375:90 , srv2086376601:90 , srv2125125674:90 , srv235750987:90 , srv258342316:90 , srv322308056:90 , srv426469882:90 , srv464459456:90 , srv474506454:90 , srv54079576:90 , srv578911871:90 , srv668982758:90 , srv714320866:90 , srv794754819:90 , srv795449561:90 , srv798785744:90 }
2023-03-10 17:56:22,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-10 17:56:22,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1464509088=6, srv132677575=4, srv714320866=26, srv1282662896=2, srv474506454=22, srv322308056=19, srv1790317296=12, srv1478690897=7, srv2086376601=15, srv668982758=25, srv1631180376=8, srv258342316=18, srv235750987=17, srv1689451363=11, srv795449561=28, srv1056402053=0, srv2032415375=14, srv1648081851=10, srv1990019523=13, srv1088126683=1, srv134068499=5, srv578911871=24, srv54079576=23, srv464459456=21, srv794754819=27, srv1302632651=3, srv2125125674=16, srv798785744=29, srv426469882=20, srv1637442471=9} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-10 17:56:22,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-10 17:56:22,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-10 17:56:22,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-10 17:56:22,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-10 17:56:22,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-10 17:56:22,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-10 17:56:22,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-10 17:56:22,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-10 17:56:22,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-10 17:56:22,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-10 17:56:22,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-10 17:56:22,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-10 17:56:22,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-10 17:56:22,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-10 17:56:22,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-10 17:56:22,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-10 17:56:22,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-10 17:56:22,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-10 17:56:22,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-10 17:56:22,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-10 17:56:22,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-10 17:56:22,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-10 17:56:22,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-10 17:56:22,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-10 17:56:22,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-10 17:56:22,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-10 17:56:22,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-10 17:56:22,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-10 17:56:22,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-10 17:56:22,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-10 17:56:22,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-03-10 17:56:22,216 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-10 17:56:22,216 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1056402053:90 , srv1088126683:90 , srv1282662896:90 , srv1302632651:90 , srv132677575:90 , srv134068499:90 , srv1464509088:90 , srv1478690897:90 , srv1631180376:90 , srv1637442471:90 , srv1648081851:90 , srv1689451363:90 , srv1790317296:90 , srv1990019523:90 , srv2032415375:90 , srv2086376601:90 , srv2125125674:90 , srv235750987:90 , srv258342316:90 , srv322308056:90 , srv426469882:90 , srv464459456:90 , srv474506454:90 , srv54079576:90 , srv578911871:90 , srv668982758:90 , srv714320866:90 , srv794754819:90 , srv795449561:90 , srv798785744:90 }
2023-03-10 17:56:22,231 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=158 (was 328), ProcessCount=170 (was 170), AvailableMemoryMB=5420 (was 5706)