2023-11-18 12:55:15,129 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-11-18 12:55:15,402 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-11-18 12:55:15,501 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-11-18 12:55:15,558 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-18 12:55:15,565 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-11-18 12:55:15,577 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=135, ProcessCount=168, AvailableMemoryMB=22037
2023-11-18 12:55:15,579 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-18 12:55:15,579 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-11-18 12:55:15,691 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1050921518:84 , srv1065917854:84 , srv1076178615:84 , srv1085380644:84 , srv1098526326:84 , srv1108071380:84 , srv1133856336:84 , srv115461631:84 , srv1184277661:84 , srv119097327:84 , srv120592918:84 , srv1238139115:84 , srv1263965912:84 , srv1302369723:84 , srv1385812282:84 , srv1398220988:84 , srv1410922973:84 , srv1418946349:84 , srv1457379050:84 , srv1462108556:84 , srv1472733753:84 , srv1488846854:84 , srv1536767228:84 , srv1565998711:84 , srv1573895283:84 , srv1590252721:84 , srv1597098957:84 , srv1601487600:84 , srv1615235831:84 , srv1636855080:84 , srv1636869097:84 , srv1643012449:84 , srv1645627193:84 , srv1665532841:84 , srv1681906840:84 , srv1701532447:84 , srv1725691917:84 , srv1760745160:84 , srv1779098324:84 , srv1792254449:84 , srv1877624332:84 , srv1917797479:84 , srv1927629435:84 , srv1929660879:84 , srv1941202202:84 , srv1948285065:84 , srv19670540:84 , srv199433502:84 , srv2007793438:84 , srv2019808054:84 , srv2069460862:84 , srv2085440225:84 , srv2087457402:84 , srv2110221089:84 , srv2125355744:84 , srv2128288585:84 , srv295492646:84 , srv306216972:84 , srv320866526:84 , srv329904450:84 , srv330356279:84 , srv37136640:84 , srv374246964:84 , srv379149077:84 , srv397611466:84 , srv422445016:84 , srv422928128:84 , srv434711049:84 , srv445061608:84 , srv455392857:84 , srv486631796:84 , srv494647327:84 , srv517285955:84 , srv577742427:84 , srv578385039:84 , srv603970708:84 , srv634324108:84 , srv641711842:84 , srv650372886:84 , srv733480975:84 , srv759392247:84 , srv778842605:84 , srv797262327:84 , srv798326651:84 , srv803763424:84 , srv815407484:84 , srv829224054:84 , srv838778198:84 , srv839834259:84 , srv853993371:84 , srv874449085:84 , srv908066178:84 , srv920590984:84 , srv929934618:84 , srv936861860:84 , srv951648216:84 , srv956172288:84 , srv965889522:84 , srv975624825:84 , srv2023561543:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-11-18 12:55:15,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-18 12:55:15,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv379149077=64, srv1098526326=4, srv1590252721=25, srv1645627193=32, srv1636855080=29, srv1385812282=14, srv445061608=69, srv1941202202=44, srv2019808054=49, srv1085380644=3, srv1636869097=30, srv1565998711=23, srv2069460862=51, srv1877624332=40, srv839834259=89, srv603970708=76, srv115461631=7, srv1779098324=38, srv1929660879=43, srv577742427=74, srv1917797479=41, srv1263965912=12, srv306216972=58, srv1462108556=19, srv19670540=46, srv1927629435=42, srv120592918=10, srv1701532447=35, srv1457379050=18, srv908066178=92, srv1076178615=2, srv929934618=94, srv1488846854=21, srv1643012449=31, srv1065917854=1, srv2023561543=50, srv1601487600=27, srv965889522=98, srv874449085=91, srv1615235831=28, srv829224054=87, srv797262327=83, srv37136640=62, srv455392857=70, srv650372886=79, srv733480975=80, srv422928128=67, srv1948285065=45, srv330356279=61, srv634324108=77, srv1410922973=16, srv2128288585=56, srv951648216=96, srv295492646=57, srv1472733753=20, srv936861860=95, srv1108071380=5, srv798326651=84, srv374246964=63, srv397611466=65, srv2110221089=54, srv1573895283=24, srv329904450=60, srv759392247=81, srv199433502=47, srv1597098957=26, srv1050921518=0, srv494647327=72, srv1302369723=13, srv1133856336=6, srv920590984=93, srv975624825=99, srv1536767228=22, srv956172288=97, srv838778198=88, srv422445016=66, srv1760745160=37, srv434711049=68, srv1238139115=11, srv1418946349=17, srv1725691917=36, srv486631796=71, srv1665532841=33, srv2087457402=53, srv2085440225=52, srv803763424=85, srv1681906840=34, srv1792254449=39, srv853993371=90, srv815407484=86, srv1184277661=8, srv778842605=82, srv320866526=59, srv517285955=73, srv119097327=9, srv2007793438=48, srv578385039=75, srv641711842=78, srv1398220988=15, srv2125355744=55} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-18 12:55:15,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-18 12:55:15,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-18 12:55:15,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-18 12:55:15,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-18 12:55:15,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-18 12:55:15,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-18 12:55:15,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-18 12:55:15,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-18 12:55:15,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-18 12:55:15,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-18 12:55:15,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-18 12:55:15,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-18 12:55:15,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-18 12:55:15,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-18 12:55:15,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-18 12:55:15,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-18 12:55:15,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-18 12:55:15,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-18 12:55:15,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-18 12:55:15,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-18 12:55:15,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-18 12:55:15,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-18 12:55:15,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-18 12:55:15,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-18 12:55:15,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-18 12:55:15,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-18 12:55:15,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-18 12:55:15,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-18 12:55:15,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-18 12:55:15,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-18 12:55:15,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-18 12:55:15,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-18 12:55:15,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-18 12:55:15,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-18 12:55:15,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-18 12:55:15,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-18 12:55:15,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-18 12:55:15,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-18 12:55:15,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-18 12:55:15,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-18 12:55:15,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-18 12:55:15,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-18 12:55:15,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-18 12:55:15,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-18 12:55:15,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-18 12:55:15,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-18 12:55:15,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-18 12:55:15,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-18 12:55:15,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-18 12:55:15,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-18 12:55:15,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-18 12:55:15,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-18 12:55:15,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-18 12:55:15,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-18 12:55:15,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-18 12:55:15,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-18 12:55:15,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-18 12:55:15,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-18 12:55:15,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-18 12:55:15,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-18 12:55:15,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-18 12:55:15,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-18 12:55:15,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-18 12:55:15,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-18 12:55:15,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-18 12:55:15,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-18 12:55:15,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-18 12:55:15,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-18 12:55:15,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-18 12:55:15,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-18 12:55:15,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-18 12:55:15,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-18 12:55:15,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-18 12:55:15,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-18 12:55:15,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-18 12:55:15,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-18 12:55:15,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-18 12:55:15,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-18 12:55:15,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-18 12:55:15,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-18 12:55:15,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-18 12:55:15,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-18 12:55:15,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-18 12:55:15,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-18 12:55:15,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-18 12:55:15,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-18 12:55:15,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-18 12:55:15,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-18 12:55:15,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-18 12:55:15,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-18 12:55:15,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-18 12:55:15,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-18 12:55:15,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-18 12:55:15,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-18 12:55:15,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-18 12:55:15,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-18 12:55:15,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-18 12:55:15,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-18 12:55:15,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-18 12:55:15,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-18 12:55:15,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-18 12:55:15,951 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-18 12:55:15,952 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-18 12:55:15,952 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-18 12:55:16,413 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 610 ms to try 5000 different iterations.  Found a solution that moves 4690 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1779260165949855. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03926783465202957, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.052664152009182105, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5211111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03926783465202957, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17291666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24208333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-18 12:55:16,423 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1184277661:81 , srv379149077:81 , srv874449085:81 , srv1760745160:82 , srv486631796:82 , srv517285955:82 , srv1601487600:83 , srv1877624332:83 , srv2125355744:83 , srv397611466:83 , srv1050921518:84 , srv1410922973:84 , srv1615235831:84 , srv1927629435:84 , srv306216972:84 , srv37136640:84 , srv603970708:84 , srv965889522:84 , srv1398220988:85 , srv1536767228:85 , srv1636869097:85 , srv1643012449:85 , srv1645627193:85 , srv1701532447:85 , srv1917797479:85 , srv2007793438:85 , srv2087457402:85 , srv295492646:85 , srv422928128:85 , srv434711049:85 , srv803763424:85 , srv853993371:85 , srv920590984:85 , srv936861860:85 , srv951648216:85 , srv1076178615:86 , srv1085380644:86 , srv1098526326:86 , srv1665532841:86 , srv1681906840:86 , srv1941202202:86 , srv2085440225:86 , srv2128288585:86 , srv329904450:86 , srv455392857:86 , srv494647327:86 , srv634324108:86 , srv650372886:86 , srv759392247:86 , srv798326651:86 , srv815407484:86 , srv838778198:86 , srv1108071380:87 , srv115461631:87 , srv120592918:87 , srv1238139115:87 , srv1385812282:87 , srv1488846854:87 , srv1573895283:87 , srv1636855080:87 , srv1725691917:87 , srv1792254449:87 , srv1929660879:87 , srv199433502:87 , srv2110221089:87 , srv330356279:87 , srv422445016:87 , srv577742427:87 , srv733480975:87 , srv829224054:87 , srv1133856336:88 , srv1263965912:88 , srv1418946349:88 , srv1462108556:88 , srv1565998711:88 , srv1590252721:88 , srv374246964:88 , srv445061608:88 , srv839834259:88 , srv1065917854:89 , srv119097327:89 , srv1472733753:89 , srv1597098957:89 , srv1779098324:89 , srv2019808054:89 , srv778842605:89 , srv956172288:89 , srv1302369723:90 , srv1948285065:90 , srv2069460862:90 , srv320866526:90 , srv578385039:90 , srv641711842:90 , srv1457379050:91 , srv908066178:91 , srv929934618:91 , srv797262327:92 , srv975624825:92 , srv19670540:94 , srv2023561543:439 }
2023-11-18 12:55:16,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-18 12:55:16,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv379149077=64, srv1098526326=4, srv1590252721=25, srv1645627193=32, srv1636855080=29, srv1385812282=14, srv445061608=69, srv1941202202=44, srv2019808054=49, srv1085380644=3, srv1636869097=30, srv1565998711=23, srv2069460862=51, srv1877624332=40, srv839834259=89, srv603970708=76, srv115461631=7, srv1779098324=38, srv1929660879=43, srv577742427=74, srv1917797479=41, srv1263965912=12, srv306216972=58, srv1462108556=19, srv19670540=46, srv1927629435=42, srv120592918=10, srv1701532447=35, srv1457379050=18, srv908066178=92, srv1076178615=2, srv929934618=94, srv1488846854=21, srv1643012449=31, srv1065917854=1, srv2023561543=50, srv1601487600=27, srv965889522=98, srv874449085=91, srv1615235831=28, srv829224054=87, srv797262327=83, srv37136640=62, srv455392857=70, srv650372886=79, srv733480975=80, srv422928128=67, srv1948285065=45, srv330356279=61, srv634324108=77, srv1410922973=16, srv2128288585=56, srv951648216=96, srv295492646=57, srv1472733753=20, srv936861860=95, srv1108071380=5, srv798326651=84, srv374246964=63, srv397611466=65, srv2110221089=54, srv1573895283=24, srv329904450=60, srv759392247=81, srv199433502=47, srv1597098957=26, srv1050921518=0, srv494647327=72, srv1302369723=13, srv1133856336=6, srv920590984=93, srv975624825=99, srv1536767228=22, srv956172288=97, srv838778198=88, srv422445016=66, srv1760745160=37, srv434711049=68, srv1238139115=11, srv1418946349=17, srv1725691917=36, srv486631796=71, srv1665532841=33, srv2087457402=53, srv2085440225=52, srv803763424=85, srv1681906840=34, srv1792254449=39, srv853993371=90, srv815407484=86, srv1184277661=8, srv778842605=82, srv320866526=59, srv517285955=73, srv119097327=9, srv2007793438=48, srv578385039=75, srv641711842=78, srv1398220988=15, srv2125355744=55} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-18 12:55:16,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-18 12:55:16,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-18 12:55:16,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-18 12:55:16,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-18 12:55:16,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-18 12:55:16,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-18 12:55:16,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-18 12:55:16,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-18 12:55:16,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-18 12:55:16,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-18 12:55:16,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-18 12:55:16,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-18 12:55:16,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-18 12:55:16,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-18 12:55:16,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-18 12:55:16,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-18 12:55:16,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-18 12:55:16,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-18 12:55:16,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-18 12:55:16,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-18 12:55:16,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-18 12:55:16,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-18 12:55:16,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-18 12:55:16,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-18 12:55:16,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-18 12:55:16,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-18 12:55:16,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-18 12:55:16,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-18 12:55:16,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-18 12:55:16,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-18 12:55:16,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-18 12:55:16,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-18 12:55:16,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-18 12:55:16,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-18 12:55:16,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-18 12:55:16,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-18 12:55:16,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-18 12:55:16,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-18 12:55:16,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-18 12:55:16,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-18 12:55:16,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-18 12:55:16,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-18 12:55:16,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-18 12:55:16,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-18 12:55:16,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-18 12:55:16,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-18 12:55:16,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-18 12:55:16,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-18 12:55:16,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-18 12:55:16,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-18 12:55:16,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-18 12:55:16,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-18 12:55:16,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-18 12:55:16,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-18 12:55:16,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-18 12:55:16,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-18 12:55:16,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-18 12:55:16,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-18 12:55:16,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-18 12:55:16,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-18 12:55:16,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-18 12:55:16,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-18 12:55:16,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-18 12:55:16,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-18 12:55:16,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-18 12:55:16,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-18 12:55:16,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-18 12:55:16,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-18 12:55:16,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-18 12:55:16,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-18 12:55:16,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-18 12:55:16,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-18 12:55:16,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-18 12:55:16,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-18 12:55:16,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-18 12:55:16,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-18 12:55:16,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-18 12:55:16,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-18 12:55:16,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-18 12:55:16,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-18 12:55:16,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-18 12:55:16,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-18 12:55:16,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-18 12:55:16,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-18 12:55:16,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-18 12:55:16,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-18 12:55:16,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-18 12:55:16,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-18 12:55:16,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-18 12:55:16,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-18 12:55:16,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-18 12:55:16,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-18 12:55:16,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-18 12:55:16,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-18 12:55:16,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-18 12:55:16,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-18 12:55:16,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-18 12:55:16,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-18 12:55:16,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-18 12:55:16,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-18 12:55:16,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-18 12:55:16,569 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03926783465202957, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.052664152009182105, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03926783465202957, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17291666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24208333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-18 12:55:16,569 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-18 12:55:16,570 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17789317651308037, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03926783465202957, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.052664152009182105, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03926783465202957, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17291666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24208333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-18 12:55:16,846 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 377 ms to try 5000 different iterations.  Found a solution that moves 2607 regions; Going from a computed imbalance of 0.17789317651308037 to a new imbalance of 0.014588252209560367. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02204304872305798); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.034863852562334756, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2896666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02204304872305798); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011416666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04475, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-18 12:55:16,855 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv19670540:77 , srv1302369723:80 , srv1948285065:80 , srv929934618:80 , srv119097327:81 , srv1457379050:81 , srv797262327:81 , srv1133856336:82 , srv1472733753:82 , srv578385039:82 , srv641711842:82 , srv908066178:82 , srv975624825:82 , srv1488846854:83 , srv1779098324:83 , srv1929660879:83 , srv2019808054:83 , srv2069460862:83 , srv320866526:83 , srv445061608:83 , srv577742427:83 , srv120592918:84 , srv1385812282:84 , srv1565998711:84 , srv1590252721:84 , srv1636855080:84 , srv1941202202:84 , srv199433502:84 , srv422445016:84 , srv650372886:84 , srv956172288:84 , srv1065917854:85 , srv115461631:85 , srv1263965912:85 , srv1665532841:85 , srv1792254449:85 , srv1917797479:85 , srv2087457402:85 , srv2110221089:85 , srv2128288585:85 , srv374246964:85 , srv778842605:85 , srv829224054:85 , srv839834259:85 , srv1050921518:86 , srv1076178615:86 , srv1098526326:86 , srv1108071380:86 , srv1398220988:86 , srv1410922973:86 , srv1418946349:86 , srv1462108556:86 , srv1573895283:86 , srv1597098957:86 , srv1725691917:86 , srv306216972:86 , srv329904450:86 , srv434711049:86 , srv455392857:86 , srv494647327:86 , srv603970708:86 , srv759392247:86 , srv815407484:86 , srv853993371:86 , srv1536767228:87 , srv1615235831:87 , srv1636869097:87 , srv1643012449:87 , srv1681906840:87 , srv2007793438:87 , srv2085440225:87 , srv295492646:87 , srv330356279:87 , srv422928128:87 , srv733480975:87 , srv798326651:87 , srv838778198:87 , srv936861860:87 , srv951648216:87 , srv1085380644:88 , srv1238139115:88 , srv1601487600:88 , srv1701532447:88 , srv1927629435:88 , srv2125355744:88 , srv634324108:88 , srv803763424:88 , srv920590984:88 , srv965889522:88 , srv1645627193:89 , srv1877624332:89 , srv37136640:89 , srv379149077:90 , srv397611466:90 , srv517285955:90 , srv1760745160:91 , srv486631796:91 , srv1184277661:92 , srv874449085:92 , srv2023561543:531 }
2023-11-18 12:55:16,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-18 12:55:16,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv379149077=64, srv1098526326=4, srv1590252721=25, srv1645627193=32, srv1636855080=29, srv1385812282=14, srv445061608=69, srv1941202202=44, srv2019808054=49, srv1085380644=3, srv1636869097=30, srv1565998711=23, srv2069460862=51, srv1877624332=40, srv839834259=89, srv603970708=76, srv115461631=7, srv1779098324=38, srv1929660879=43, srv577742427=74, srv1917797479=41, srv1263965912=12, srv306216972=58, srv1462108556=19, srv19670540=46, srv1927629435=42, srv120592918=10, srv1701532447=35, srv1457379050=18, srv908066178=92, srv1076178615=2, srv929934618=94, srv1488846854=21, srv1643012449=31, srv1065917854=1, srv2023561543=50, srv1601487600=27, srv965889522=98, srv874449085=91, srv1615235831=28, srv829224054=87, srv797262327=83, srv37136640=62, srv455392857=70, srv650372886=79, srv733480975=80, srv422928128=67, srv1948285065=45, srv330356279=61, srv634324108=77, srv1410922973=16, srv2128288585=56, srv951648216=96, srv295492646=57, srv1472733753=20, srv936861860=95, srv1108071380=5, srv798326651=84, srv374246964=63, srv397611466=65, srv2110221089=54, srv1573895283=24, srv329904450=60, srv759392247=81, srv199433502=47, srv1597098957=26, srv1050921518=0, srv494647327=72, srv1302369723=13, srv1133856336=6, srv920590984=93, srv975624825=99, srv1536767228=22, srv956172288=97, srv838778198=88, srv422445016=66, srv1760745160=37, srv434711049=68, srv1238139115=11, srv1418946349=17, srv1725691917=36, srv486631796=71, srv1665532841=33, srv2087457402=53, srv2085440225=52, srv803763424=85, srv1681906840=34, srv1792254449=39, srv853993371=90, srv815407484=86, srv1184277661=8, srv778842605=82, srv320866526=59, srv517285955=73, srv119097327=9, srv2007793438=48, srv578385039=75, srv641711842=78, srv1398220988=15, srv2125355744=55} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-18 12:55:16,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-18 12:55:16,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-18 12:55:16,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-18 12:55:16,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-18 12:55:16,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-18 12:55:16,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-18 12:55:16,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-18 12:55:16,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-18 12:55:16,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-18 12:55:16,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-18 12:55:16,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-18 12:55:16,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-18 12:55:16,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-18 12:55:16,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-18 12:55:16,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-18 12:55:16,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-18 12:55:16,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-18 12:55:16,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-18 12:55:16,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-18 12:55:16,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-18 12:55:16,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-18 12:55:16,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-18 12:55:16,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-18 12:55:16,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-18 12:55:16,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-18 12:55:16,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-18 12:55:16,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-18 12:55:16,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-18 12:55:16,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-18 12:55:16,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-18 12:55:16,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-18 12:55:16,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-18 12:55:16,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-18 12:55:16,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-18 12:55:16,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-18 12:55:16,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-18 12:55:16,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-18 12:55:16,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-18 12:55:16,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-18 12:55:16,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-18 12:55:16,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-18 12:55:16,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-18 12:55:16,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-18 12:55:16,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-18 12:55:16,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-18 12:55:16,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-18 12:55:16,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-18 12:55:16,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-18 12:55:16,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-18 12:55:16,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-18 12:55:16,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-18 12:55:16,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-18 12:55:16,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-18 12:55:16,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-18 12:55:16,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-18 12:55:16,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-18 12:55:16,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-18 12:55:16,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-18 12:55:16,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-18 12:55:16,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-18 12:55:16,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-18 12:55:16,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-18 12:55:16,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-18 12:55:16,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-18 12:55:16,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-18 12:55:16,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-18 12:55:16,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-18 12:55:16,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-18 12:55:16,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-18 12:55:16,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-18 12:55:16,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-18 12:55:16,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-18 12:55:16,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-18 12:55:16,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-18 12:55:16,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-18 12:55:16,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-18 12:55:16,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-18 12:55:16,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-18 12:55:16,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-18 12:55:16,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-18 12:55:16,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-18 12:55:16,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-18 12:55:16,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-18 12:55:16,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-18 12:55:16,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-18 12:55:16,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-18 12:55:16,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-18 12:55:16,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-18 12:55:16,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-18 12:55:16,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-18 12:55:16,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-18 12:55:16,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-18 12:55:16,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-18 12:55:16,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-18 12:55:16,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-18 12:55:16,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-18 12:55:16,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-18 12:55:16,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-18 12:55:16,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-18 12:55:16,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-18 12:55:16,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-18 12:55:17,008 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02204304872305798); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.034863852562334756, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02204304872305798); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011416666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04475, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-18 12:55:17,009 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-18 12:55:17,009 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.014569997605396888, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02204304872305798); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.034863852562334756, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02204304872305798); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011416666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04475, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-18 12:55:17,375 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 463 ms to try 5000 different iterations.  Found a solution that moves 990 regions; Going from a computed imbalance of 0.014569997605396888 to a new imbalance of 5.722914753120515E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007101424378813449); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007998316321143093); MoveCostFunction : (multiplier=7.0, imbalance=0.11, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007101424378813449); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-18 12:55:17,379 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1238139115:82 , srv1597098957:82 , srv1085380644:83 , srv1418946349:83 , srv1645627193:83 , srv330356279:83 , srv634324108:83 , srv1065917854:84 , srv1108071380:84 , srv1184277661:84 , srv1263965912:84 , srv1462108556:84 , srv1573895283:84 , srv1681906840:84 , srv2085440225:84 , srv374246964:84 , srv486631796:84 , srv494647327:84 , srv733480975:84 , srv778842605:84 , srv798326651:84 , srv803763424:84 , srv908066178:84 , srv920590984:84 , srv975624825:84 , srv1076178615:85 , srv115461631:85 , srv1398220988:85 , srv1457379050:85 , srv1536767228:85 , srv1565998711:85 , srv1590252721:85 , srv1636855080:85 , srv1643012449:85 , srv1725691917:85 , srv1760745160:85 , srv1779098324:85 , srv1792254449:85 , srv1927629435:85 , srv2069460862:85 , srv295492646:85 , srv320866526:85 , srv329904450:85 , srv37136640:85 , srv397611466:85 , srv422928128:85 , srv517285955:85 , srv578385039:85 , srv641711842:85 , srv759392247:85 , srv838778198:85 , srv839834259:85 , srv874449085:85 , srv936861860:85 , srv951648216:85 , srv956172288:85 , srv965889522:85 , srv1098526326:86 , srv120592918:86 , srv1601487600:86 , srv1615235831:86 , srv1636869097:86 , srv1665532841:86 , srv1701532447:86 , srv1877624332:86 , srv19670540:86 , srv2007793438:86 , srv2019808054:86 , srv2110221089:86 , srv2125355744:86 , srv2128288585:86 , srv379149077:86 , srv422445016:86 , srv434711049:86 , srv455392857:86 , srv797262327:86 , srv815407484:86 , srv829224054:86 , srv929934618:86 , srv1050921518:87 , srv1133856336:87 , srv119097327:87 , srv1472733753:87 , srv1917797479:87 , srv1929660879:87 , srv1948285065:87 , srv199433502:87 , srv306216972:87 , srv445061608:87 , srv577742427:87 , srv603970708:87 , srv650372886:87 , srv853993371:87 , srv1302369723:88 , srv1385812282:88 , srv1410922973:88 , srv1488846854:88 , srv1941202202:88 , srv2087457402:88 , srv2023561543:551 }
2023-11-18 12:55:17,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-18 12:55:17,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv379149077=64, srv1098526326=4, srv1590252721=25, srv1645627193=32, srv1636855080=29, srv1385812282=14, srv445061608=69, srv1941202202=44, srv2019808054=49, srv1085380644=3, srv1636869097=30, srv1565998711=23, srv2069460862=51, srv1877624332=40, srv839834259=89, srv603970708=76, srv115461631=7, srv1779098324=38, srv1929660879=43, srv577742427=74, srv1917797479=41, srv1263965912=12, srv306216972=58, srv1462108556=19, srv19670540=46, srv1927629435=42, srv120592918=10, srv1701532447=35, srv1457379050=18, srv908066178=92, srv1076178615=2, srv929934618=94, srv1488846854=21, srv1643012449=31, srv1065917854=1, srv2023561543=50, srv1601487600=27, srv965889522=98, srv874449085=91, srv1615235831=28, srv829224054=87, srv797262327=83, srv37136640=62, srv455392857=70, srv650372886=79, srv733480975=80, srv422928128=67, srv1948285065=45, srv330356279=61, srv634324108=77, srv1410922973=16, srv2128288585=56, srv951648216=96, srv295492646=57, srv1472733753=20, srv936861860=95, srv1108071380=5, srv798326651=84, srv374246964=63, srv397611466=65, srv2110221089=54, srv1573895283=24, srv329904450=60, srv759392247=81, srv199433502=47, srv1597098957=26, srv1050921518=0, srv494647327=72, srv1302369723=13, srv1133856336=6, srv920590984=93, srv975624825=99, srv1536767228=22, srv956172288=97, srv838778198=88, srv422445016=66, srv1760745160=37, srv434711049=68, srv1238139115=11, srv1418946349=17, srv1725691917=36, srv486631796=71, srv1665532841=33, srv2087457402=53, srv2085440225=52, srv803763424=85, srv1681906840=34, srv1792254449=39, srv853993371=90, srv815407484=86, srv1184277661=8, srv778842605=82, srv320866526=59, srv517285955=73, srv119097327=9, srv2007793438=48, srv578385039=75, srv641711842=78, srv1398220988=15, srv2125355744=55} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-18 12:55:17,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-18 12:55:17,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-18 12:55:17,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-18 12:55:17,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-18 12:55:17,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-18 12:55:17,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-18 12:55:17,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-18 12:55:17,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-18 12:55:17,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-18 12:55:17,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-18 12:55:17,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-18 12:55:17,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-18 12:55:17,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-18 12:55:17,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-18 12:55:17,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-18 12:55:17,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-18 12:55:17,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-18 12:55:17,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-18 12:55:17,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-18 12:55:17,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-18 12:55:17,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-18 12:55:17,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-18 12:55:17,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-18 12:55:17,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-18 12:55:17,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-18 12:55:17,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-18 12:55:17,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-18 12:55:17,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-18 12:55:17,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-18 12:55:17,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-18 12:55:17,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-18 12:55:17,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-18 12:55:17,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-18 12:55:17,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-18 12:55:17,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-18 12:55:17,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-18 12:55:17,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-18 12:55:17,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-18 12:55:17,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-18 12:55:17,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-18 12:55:17,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-18 12:55:17,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-18 12:55:17,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-18 12:55:17,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-18 12:55:17,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-18 12:55:17,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-18 12:55:17,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-18 12:55:17,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-18 12:55:17,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-18 12:55:17,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-18 12:55:17,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-18 12:55:17,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-18 12:55:17,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-18 12:55:17,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-18 12:55:17,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-18 12:55:17,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-18 12:55:17,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-18 12:55:17,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-18 12:55:17,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-18 12:55:17,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-18 12:55:17,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-18 12:55:17,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-18 12:55:17,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-18 12:55:17,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-18 12:55:17,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-18 12:55:17,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-18 12:55:17,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-18 12:55:17,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-18 12:55:17,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-18 12:55:17,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-18 12:55:17,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-18 12:55:17,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-18 12:55:17,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-18 12:55:17,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-18 12:55:17,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-18 12:55:17,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-18 12:55:17,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-18 12:55:17,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-18 12:55:17,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-18 12:55:17,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-18 12:55:17,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-18 12:55:17,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-18 12:55:17,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-18 12:55:17,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-18 12:55:17,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-18 12:55:17,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-18 12:55:17,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-18 12:55:17,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-18 12:55:17,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-18 12:55:17,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-18 12:55:17,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-18 12:55:17,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-18 12:55:17,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-18 12:55:17,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-18 12:55:17,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-18 12:55:17,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-18 12:55:17,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-18 12:55:17,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-18 12:55:17,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-18 12:55:17,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-18 12:55:17,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-18 12:55:17,508 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007101424378813449); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007998316321143093); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007101424378813449); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-18 12:55:17,509 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-18 12:55:17,509 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=5.653593471487053E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007101424378813449); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007998316321143093); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007101424378813449); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-18 12:55:17,804 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 391 ms to try 5000 different iterations.  Found a solution that moves 268 regions; Going from a computed imbalance of 5.653593471487053E-4 to a new imbalance of 2.0914629997393907E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.694119368783758E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0019534423844256277); MoveCostFunction : (multiplier=7.0, imbalance=0.029777777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.694119368783758E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-18 12:55:17,805 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1385812282:83 , srv641711842:83 , srv797262327:83 , srv815407484:83 , srv1065917854:84 , srv1133856336:84 , srv115461631:84 , srv1263965912:84 , srv1302369723:84 , srv1410922973:84 , srv1462108556:84 , srv1472733753:84 , srv1488846854:84 , srv1536767228:84 , srv1636869097:84 , srv1701532447:84 , srv1725691917:84 , srv1760745160:84 , srv1877624332:84 , srv1927629435:84 , srv1941202202:84 , srv199433502:84 , srv2007793438:84 , srv2069460862:84 , srv2087457402:84 , srv2110221089:84 , srv37136640:84 , srv397611466:84 , srv422445016:84 , srv445061608:84 , srv455392857:84 , srv577742427:84 , srv603970708:84 , srv778842605:84 , srv838778198:84 , srv839834259:84 , srv853993371:84 , srv874449085:84 , srv956172288:84 , srv965889522:84 , srv1050921518:85 , srv1076178615:85 , srv1085380644:85 , srv1098526326:85 , srv1108071380:85 , srv1184277661:85 , srv119097327:85 , srv120592918:85 , srv1238139115:85 , srv1418946349:85 , srv1457379050:85 , srv1565998711:85 , srv1573895283:85 , srv1590252721:85 , srv1597098957:85 , srv1601487600:85 , srv1615235831:85 , srv1643012449:85 , srv1645627193:85 , srv1665532841:85 , srv1681906840:85 , srv1779098324:85 , srv1917797479:85 , srv1929660879:85 , srv1948285065:85 , srv19670540:85 , srv2019808054:85 , srv2085440225:85 , srv2125355744:85 , srv2128288585:85 , srv295492646:85 , srv306216972:85 , srv320866526:85 , srv329904450:85 , srv374246964:85 , srv379149077:85 , srv434711049:85 , srv486631796:85 , srv517285955:85 , srv650372886:85 , srv733480975:85 , srv759392247:85 , srv798326651:85 , srv803763424:85 , srv829224054:85 , srv908066178:85 , srv920590984:85 , srv929934618:85 , srv936861860:85 , srv951648216:85 , srv975624825:85 , srv1398220988:86 , srv1636855080:86 , srv1792254449:86 , srv330356279:86 , srv422928128:86 , srv494647327:86 , srv578385039:86 , srv634324108:86 , srv2023561543:621 }
2023-11-18 12:55:17,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-18 12:55:17,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv379149077=64, srv1098526326=4, srv1590252721=25, srv1645627193=32, srv1636855080=29, srv1385812282=14, srv445061608=69, srv1941202202=44, srv2019808054=49, srv1085380644=3, srv1636869097=30, srv1565998711=23, srv2069460862=51, srv1877624332=40, srv839834259=89, srv603970708=76, srv115461631=7, srv1779098324=38, srv1929660879=43, srv577742427=74, srv1917797479=41, srv1263965912=12, srv306216972=58, srv1462108556=19, srv19670540=46, srv1927629435=42, srv120592918=10, srv1701532447=35, srv1457379050=18, srv908066178=92, srv1076178615=2, srv929934618=94, srv1488846854=21, srv1643012449=31, srv1065917854=1, srv2023561543=50, srv1601487600=27, srv965889522=98, srv874449085=91, srv1615235831=28, srv829224054=87, srv797262327=83, srv37136640=62, srv455392857=70, srv650372886=79, srv733480975=80, srv422928128=67, srv1948285065=45, srv330356279=61, srv634324108=77, srv1410922973=16, srv2128288585=56, srv951648216=96, srv295492646=57, srv1472733753=20, srv936861860=95, srv1108071380=5, srv798326651=84, srv374246964=63, srv397611466=65, srv2110221089=54, srv1573895283=24, srv329904450=60, srv759392247=81, srv199433502=47, srv1597098957=26, srv1050921518=0, srv494647327=72, srv1302369723=13, srv1133856336=6, srv920590984=93, srv975624825=99, srv1536767228=22, srv956172288=97, srv838778198=88, srv422445016=66, srv1760745160=37, srv434711049=68, srv1238139115=11, srv1418946349=17, srv1725691917=36, srv486631796=71, srv1665532841=33, srv2087457402=53, srv2085440225=52, srv803763424=85, srv1681906840=34, srv1792254449=39, srv853993371=90, srv815407484=86, srv1184277661=8, srv778842605=82, srv320866526=59, srv517285955=73, srv119097327=9, srv2007793438=48, srv578385039=75, srv641711842=78, srv1398220988=15, srv2125355744=55} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-18 12:55:17,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-18 12:55:17,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-18 12:55:17,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-18 12:55:17,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-18 12:55:17,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-18 12:55:17,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-18 12:55:17,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-18 12:55:17,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-18 12:55:17,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-18 12:55:17,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-18 12:55:17,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-18 12:55:17,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-18 12:55:17,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-18 12:55:17,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-18 12:55:17,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-18 12:55:17,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-18 12:55:17,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-18 12:55:17,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-18 12:55:17,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-18 12:55:17,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-18 12:55:17,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-18 12:55:17,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-18 12:55:17,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-18 12:55:17,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-18 12:55:17,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-18 12:55:17,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-18 12:55:17,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-18 12:55:17,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-18 12:55:17,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-18 12:55:17,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-18 12:55:17,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-18 12:55:17,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-18 12:55:17,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-18 12:55:17,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-18 12:55:17,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-18 12:55:17,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-18 12:55:17,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-18 12:55:17,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-18 12:55:17,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-18 12:55:17,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-18 12:55:17,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-18 12:55:17,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-18 12:55:17,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-18 12:55:17,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-18 12:55:17,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-18 12:55:17,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-18 12:55:17,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-18 12:55:17,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-18 12:55:17,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-18 12:55:17,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-18 12:55:17,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-18 12:55:17,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-18 12:55:17,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-18 12:55:17,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-18 12:55:17,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-18 12:55:17,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-18 12:55:17,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-18 12:55:17,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-18 12:55:17,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-18 12:55:17,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-18 12:55:17,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-18 12:55:17,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-18 12:55:17,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-18 12:55:17,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-18 12:55:17,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-18 12:55:17,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-18 12:55:17,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-18 12:55:17,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-18 12:55:17,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-18 12:55:17,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-18 12:55:17,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-18 12:55:17,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-18 12:55:17,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-18 12:55:17,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-18 12:55:17,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-18 12:55:17,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-18 12:55:17,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-18 12:55:17,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-18 12:55:17,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-18 12:55:17,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-18 12:55:17,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-18 12:55:17,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-18 12:55:17,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-18 12:55:17,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-18 12:55:17,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-18 12:55:17,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-18 12:55:17,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-18 12:55:17,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-18 12:55:17,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-18 12:55:17,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-18 12:55:17,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-18 12:55:17,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-18 12:55:17,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-18 12:55:17,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-18 12:55:17,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-18 12:55:17,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-18 12:55:17,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-18 12:55:17,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-18 12:55:17,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-18 12:55:17,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-18 12:55:17,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-18 12:55:17,950 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.903805388852848E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.694119368783758E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0019534423844256277); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.694119368783758E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-18 12:55:17,951 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1385812282:83 , srv641711842:83 , srv797262327:83 , srv815407484:83 , srv1065917854:84 , srv1133856336:84 , srv115461631:84 , srv1263965912:84 , srv1302369723:84 , srv1410922973:84 , srv1462108556:84 , srv1472733753:84 , srv1488846854:84 , srv1536767228:84 , srv1636869097:84 , srv1701532447:84 , srv1725691917:84 , srv1760745160:84 , srv1877624332:84 , srv1927629435:84 , srv1941202202:84 , srv199433502:84 , srv2007793438:84 , srv2069460862:84 , srv2087457402:84 , srv2110221089:84 , srv37136640:84 , srv397611466:84 , srv422445016:84 , srv445061608:84 , srv455392857:84 , srv577742427:84 , srv603970708:84 , srv778842605:84 , srv838778198:84 , srv839834259:84 , srv853993371:84 , srv874449085:84 , srv956172288:84 , srv965889522:84 , srv1050921518:85 , srv1076178615:85 , srv1085380644:85 , srv1098526326:85 , srv1108071380:85 , srv1184277661:85 , srv119097327:85 , srv120592918:85 , srv1238139115:85 , srv1418946349:85 , srv1457379050:85 , srv1565998711:85 , srv1573895283:85 , srv1590252721:85 , srv1597098957:85 , srv1601487600:85 , srv1615235831:85 , srv1643012449:85 , srv1645627193:85 , srv1665532841:85 , srv1681906840:85 , srv1779098324:85 , srv1917797479:85 , srv1929660879:85 , srv1948285065:85 , srv19670540:85 , srv2019808054:85 , srv2085440225:85 , srv2125355744:85 , srv2128288585:85 , srv295492646:85 , srv306216972:85 , srv320866526:85 , srv329904450:85 , srv374246964:85 , srv379149077:85 , srv434711049:85 , srv486631796:85 , srv517285955:85 , srv650372886:85 , srv733480975:85 , srv759392247:85 , srv798326651:85 , srv803763424:85 , srv829224054:85 , srv908066178:85 , srv920590984:85 , srv929934618:85 , srv936861860:85 , srv951648216:85 , srv975624825:85 , srv1398220988:86 , srv1636855080:86 , srv1792254449:86 , srv330356279:86 , srv422928128:86 , srv494647327:86 , srv578385039:86 , srv634324108:86 , srv2023561543:621 }
2023-11-18 12:55:17,981 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=135 (was 135), ProcessCount=168 (was 168), AvailableMemoryMB=21526 (was 22037)
2023-11-18 12:55:17,990 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=135, ProcessCount=168, AvailableMemoryMB=21525
2023-11-18 12:55:17,991 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-18 12:55:17,991 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-11-18 12:55:18,001 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1093115533:84 , srv1107482330:84 , srv1134072662:84 , srv1162357791:84 , srv136311620:84 , srv143253662:84 , srv1509894362:84 , srv1568815493:84 , srv1746769520:84 , srv176144268:84 , srv1804875192:84 , srv1820245617:84 , srv1867564015:84 , srv204241717:84 , srv26879451:84 , srv279204108:84 , srv289164253:84 , srv341850161:84 , srv397256419:84 , srv438112884:84 , srv48322979:84 , srv510038924:84 , srv55486838:84 , srv687746045:84 , srv689556375:84 , srv703271005:84 , srv718953883:84 , srv798352517:84 , srv800173893:84 , srv1152479342:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-11-18 12:55:18,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-18 12:55:18,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv176144268=10, srv1867564015=13, srv687746045=24, srv1509894362=7, srv1804875192=11, srv510038924=22, srv26879451=15, srv289164253=17, srv1134072662=2, srv279204108=16, srv800173893=29, srv341850161=18, srv1568815493=8, srv1107482330=1, srv1162357791=4, srv438112884=20, srv136311620=5, srv55486838=23, srv1152479342=3, srv204241717=14, srv397256419=19, srv143253662=6, srv1820245617=12, srv689556375=25, srv718953883=27, srv703271005=26, srv48322979=21, srv798352517=28, srv1746769520=9, srv1093115533=0} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-18 12:55:18,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-18 12:55:18,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-18 12:55:18,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-18 12:55:18,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-18 12:55:18,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-18 12:55:18,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-18 12:55:18,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-18 12:55:18,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-18 12:55:18,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-18 12:55:18,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-18 12:55:18,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-18 12:55:18,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-18 12:55:18,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-18 12:55:18,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-18 12:55:18,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-18 12:55:18,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-18 12:55:18,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-18 12:55:18,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-18 12:55:18,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-18 12:55:18,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-18 12:55:18,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-18 12:55:18,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-18 12:55:18,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-18 12:55:18,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-18 12:55:18,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-18 12:55:18,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-18 12:55:18,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-18 12:55:18,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-18 12:55:18,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-18 12:55:18,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-18 12:55:18,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-11-18 12:55:18,026 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1110978490716623, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-18 12:55:18,026 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912543108959025, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1110978490716623, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-11-18 12:57:18,015 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5684046 different iterations.  Found a solution that moves 1856 regions; Going from a computed imbalance of 0.9912543108959025 to a new imbalance of 4.3319965896196796E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6874074074074074, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-18 12:57:18,021 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1093115533:90 , srv1107482330:90 , srv1134072662:90 , srv1152479342:90 , srv1162357791:90 , srv136311620:90 , srv143253662:90 , srv1509894362:90 , srv1568815493:90 , srv1746769520:90 , srv176144268:90 , srv1804875192:90 , srv1820245617:90 , srv1867564015:90 , srv204241717:90 , srv26879451:90 , srv279204108:90 , srv289164253:90 , srv341850161:90 , srv397256419:90 , srv438112884:90 , srv48322979:90 , srv510038924:90 , srv55486838:90 , srv687746045:90 , srv689556375:90 , srv703271005:90 , srv718953883:90 , srv798352517:90 , srv800173893:90 }
2023-11-18 12:57:18,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-18 12:57:18,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv176144268=10, srv1867564015=13, srv687746045=24, srv1509894362=7, srv1804875192=11, srv510038924=22, srv26879451=15, srv289164253=17, srv1134072662=2, srv279204108=16, srv800173893=29, srv341850161=18, srv1568815493=8, srv1107482330=1, srv1162357791=4, srv438112884=20, srv136311620=5, srv55486838=23, srv1152479342=3, srv204241717=14, srv397256419=19, srv143253662=6, srv1820245617=12, srv689556375=25, srv718953883=27, srv703271005=26, srv48322979=21, srv798352517=28, srv1746769520=9, srv1093115533=0} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-18 12:57:18,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-18 12:57:18,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-18 12:57:18,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-18 12:57:18,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-18 12:57:18,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-18 12:57:18,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-18 12:57:18,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-18 12:57:18,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-18 12:57:18,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-18 12:57:18,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-18 12:57:18,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-18 12:57:18,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-18 12:57:18,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-18 12:57:18,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-18 12:57:18,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-18 12:57:18,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-18 12:57:18,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-18 12:57:18,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-18 12:57:18,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-18 12:57:18,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-18 12:57:18,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-18 12:57:18,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-18 12:57:18,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-18 12:57:18,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-18 12:57:18,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-18 12:57:18,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-18 12:57:18,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-18 12:57:18,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-18 12:57:18,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-18 12:57:18,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-18 12:57:18,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-11-18 12:57:18,042 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-18 12:57:18,042 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1093115533:90 , srv1107482330:90 , srv1134072662:90 , srv1152479342:90 , srv1162357791:90 , srv136311620:90 , srv143253662:90 , srv1509894362:90 , srv1568815493:90 , srv1746769520:90 , srv176144268:90 , srv1804875192:90 , srv1820245617:90 , srv1867564015:90 , srv204241717:90 , srv26879451:90 , srv279204108:90 , srv289164253:90 , srv341850161:90 , srv397256419:90 , srv438112884:90 , srv48322979:90 , srv510038924:90 , srv55486838:90 , srv687746045:90 , srv689556375:90 , srv703271005:90 , srv718953883:90 , srv798352517:90 , srv800173893:90 }
2023-11-18 12:57:18,057 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=175 (was 135) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=21743 (was 21525) - AvailableMemoryMB LEAK? -