2023-05-22 13:53:31,446 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-05-22 13:53:31,771 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-05-22 13:53:31,886 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-05-22 13:53:31,950 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-22 13:53:31,957 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-05-22 13:53:31,971 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=264, MaxFileDescriptor=60000, SystemLoadAverage=321, ProcessCount=171, AvailableMemoryMB=7343
2023-05-22 13:53:31,974 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-22 13:53:31,974 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-05-22 13:53:32,123 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1007670605:84 , srv1011376198:84 , srv1059706113:84 , srv106136280:84 , srv1073701576:84 , srv1107569474:84 , srv1126228752:84 , srv1144262500:84 , srv1146542352:84 , srv1163445478:84 , srv1194623599:84 , srv1199927594:84 , srv1203858072:84 , srv1222833994:84 , srv1273404905:84 , srv1278288115:84 , srv130078235:84 , srv1322605505:84 , srv137677414:84 , srv1384597321:84 , srv138684502:84 , srv1416347829:84 , srv1444276069:84 , srv1472363115:84 , srv1474366233:84 , srv151727304:84 , srv1529845314:84 , srv1628915017:84 , srv1688921318:84 , srv1702188711:84 , srv1724972915:84 , srv1745437300:84 , srv1747089101:84 , srv1749405607:84 , srv1753840478:84 , srv1767215336:84 , srv1775862568:84 , srv1776884988:84 , srv1778046115:84 , srv1798776872:84 , srv1820724302:84 , srv1841130723:84 , srv1866490945:84 , srv1882119651:84 , srv1888849886:84 , srv1892502629:84 , srv1896849683:84 , srv190450371:84 , srv1910005333:84 , srv1921138824:84 , srv1937293531:84 , srv1940321886:84 , srv1954436782:84 , srv1997364286:84 , srv2015981010:84 , srv2061364137:84 , srv2100350622:84 , srv2114136515:84 , srv2142906500:84 , srv221575524:84 , srv234283587:84 , srv240649964:84 , srv260624916:84 , srv269817899:84 , srv275523254:84 , srv306879216:84 , srv309357591:84 , srv311005296:84 , srv404578076:84 , srv417263364:84 , srv418532174:84 , srv437808286:84 , srv458234146:84 , srv492993951:84 , srv509148311:84 , srv519207831:84 , srv522868692:84 , srv526495924:84 , srv53665565:84 , srv544641420:84 , srv571556610:84 , srv578483542:84 , srv599956092:84 , srv650526611:84 , srv653551651:84 , srv692427208:84 , srv705189435:84 , srv727219709:84 , srv728267787:84 , srv755549421:84 , srv781282988:84 , srv79040743:84 , srv80889046:84 , srv820813025:84 , srv858235094:84 , srv873339616:84 , srv924926808:84 , srv941064531:84 , srv968793356:84 , srv651781517:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-05-22 13:53:32,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-22 13:53:32,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv941064531=98, srv1866490945=42, srv1163445478=9, srv151727304=25, srv1322605505=17, srv1203858072=12, srv1472363115=23, srv234283587=60, srv1222833994=13, srv269817899=63, srv781282988=91, srv1688921318=28, srv1146542352=8, srv1910005333=48, srv705189435=87, srv1896849683=46, srv1775862568=36, srv1888849886=44, srv1937293531=50, srv311005296=67, srv2061364137=55, srv137677414=18, srv820813025=94, srv79040743=92, srv924926808=97, srv526495924=77, srv1778046115=38, srv1011376198=1, srv571556610=80, srv1940321886=51, srv404578076=68, srv578483542=81, srv2015981010=54, srv650526611=83, srv1416347829=21, srv1820724302=40, srv1628915017=27, srv417263364=69, srv1882119651=43, srv1273404905=14, srv1753840478=34, srv1841130723=41, srv1997364286=53, srv2142906500=58, srv653551651=85, srv306879216=65, srv692427208=86, srv1199927594=11, srv1954436782=52, srv106136280=3, srv1278288115=15, srv1776884988=37, srv519207831=75, srv309357591=66, srv1073701576=4, srv858235094=95, srv1749405607=33, srv458234146=72, srv755549421=90, srv53665565=78, srv1194623599=10, srv2100350622=56, srv1767215336=35, srv1798776872=39, srv2114136515=57, srv190450371=47, srv1724972915=30, srv1747089101=32, srv130078235=16, srv544641420=79, srv1529845314=26, srv651781517=84, srv1702188711=29, srv727219709=88, srv1745437300=31, srv1107569474=5, srv418532174=70, srv522868692=76, srv1059706113=2, srv240649964=61, srv1384597321=19, srv1444276069=22, srv1007670605=0, srv509148311=74, srv138684502=20, srv1474366233=24, srv275523254=64, srv728267787=89, srv221575524=59, srv492993951=73, srv968793356=99, srv1892502629=45, srv80889046=93, srv1144262500=7, srv1126228752=6, srv599956092=82, srv437808286=71, srv873339616=96, srv1921138824=49, srv260624916=62} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-22 13:53:32,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-22 13:53:32,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-22 13:53:32,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-22 13:53:32,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-22 13:53:32,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-22 13:53:32,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-22 13:53:32,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-22 13:53:32,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-22 13:53:32,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-22 13:53:32,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-22 13:53:32,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-22 13:53:32,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-22 13:53:32,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-22 13:53:32,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-22 13:53:32,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-22 13:53:32,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-22 13:53:32,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-22 13:53:32,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-22 13:53:32,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-22 13:53:32,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-22 13:53:32,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-22 13:53:32,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-22 13:53:32,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-22 13:53:32,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-22 13:53:32,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-22 13:53:32,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-22 13:53:32,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-22 13:53:32,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-22 13:53:32,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-22 13:53:32,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-22 13:53:32,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-22 13:53:32,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-22 13:53:32,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-22 13:53:32,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-22 13:53:32,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-22 13:53:32,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-22 13:53:32,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-22 13:53:32,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-22 13:53:32,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-22 13:53:32,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-22 13:53:32,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-22 13:53:32,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-22 13:53:32,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-22 13:53:32,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-22 13:53:32,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-22 13:53:32,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-22 13:53:32,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-22 13:53:32,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-22 13:53:32,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-22 13:53:32,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-22 13:53:32,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-22 13:53:32,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-22 13:53:32,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-22 13:53:32,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-22 13:53:32,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-22 13:53:32,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-22 13:53:32,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-22 13:53:32,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-22 13:53:32,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-22 13:53:32,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-22 13:53:32,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-22 13:53:32,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-22 13:53:32,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-22 13:53:32,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-22 13:53:32,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-22 13:53:32,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-22 13:53:32,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-22 13:53:32,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-22 13:53:32,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-22 13:53:32,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-22 13:53:32,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-22 13:53:32,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-22 13:53:32,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-22 13:53:32,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-22 13:53:32,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-22 13:53:32,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-22 13:53:32,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-22 13:53:32,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-22 13:53:32,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-22 13:53:32,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-22 13:53:32,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-22 13:53:32,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-22 13:53:32,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-22 13:53:32,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-22 13:53:32,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-22 13:53:32,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-22 13:53:32,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-22 13:53:32,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-22 13:53:32,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-22 13:53:32,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-22 13:53:32,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-22 13:53:32,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-22 13:53:32,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-22 13:53:32,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-22 13:53:32,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-22 13:53:32,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-22 13:53:32,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-22 13:53:32,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-22 13:53:32,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-22 13:53:32,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-22 13:53:32,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-22 13:53:32,668 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-22 13:53:32,668 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-22 13:53:32,668 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-22 13:53:33,069 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 786 ms to try 5000 different iterations.  Found a solution that moves 4676 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18007961048415094. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040372069281275715, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.054933895587328876, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5195555555555555, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040372069281275715, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17541666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24083333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-22 13:53:33,086 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1798776872:82 , srv190450371:82 , srv858235094:82 , srv1778046115:83 , srv1841130723:83 , srv240649964:83 , srv650526611:83 , srv80889046:83 , srv1199927594:84 , srv1322605505:84 , srv1416347829:84 , srv1753840478:84 , srv1776884988:84 , srv1820724302:84 , srv221575524:84 , srv53665565:84 , srv544641420:84 , srv653551651:84 , srv781282988:84 , srv1007670605:85 , srv1011376198:85 , srv137677414:85 , srv138684502:85 , srv1472363115:85 , srv1749405607:85 , srv2015981010:85 , srv2061364137:85 , srv2114136515:85 , srv2142906500:85 , srv269817899:85 , srv309357591:85 , srv526495924:85 , srv755549421:85 , srv820813025:85 , srv924926808:85 , srv941064531:85 , srv968793356:85 , srv1107569474:86 , srv1144262500:86 , srv1163445478:86 , srv1278288115:86 , srv1384597321:86 , srv1444276069:86 , srv151727304:86 , srv1688921318:86 , srv1745437300:86 , srv1882119651:86 , srv1997364286:86 , srv260624916:86 , srv275523254:86 , srv418532174:86 , srv509148311:86 , srv519207831:86 , srv578483542:86 , srv692427208:86 , srv727219709:86 , srv1059706113:87 , srv1222833994:87 , srv1273404905:87 , srv130078235:87 , srv1529845314:87 , srv1628915017:87 , srv1702188711:87 , srv1767215336:87 , srv1775862568:87 , srv1866490945:87 , srv1888849886:87 , srv1896849683:87 , srv1940321886:87 , srv234283587:87 , srv458234146:87 , srv492993951:87 , srv599956092:87 , srv873339616:87 , srv1724972915:88 , srv1892502629:88 , srv1954436782:88 , srv306879216:88 , srv404578076:88 , srv417263364:88 , srv437808286:88 , srv571556610:88 , srv705189435:88 , srv1126228752:89 , srv1194623599:89 , srv1910005333:89 , srv1921138824:89 , srv311005296:89 , srv728267787:89 , srv106136280:90 , srv1073701576:90 , srv1146542352:90 , srv1203858072:90 , srv1747089101:90 , srv2100350622:90 , srv79040743:91 , srv1937293531:92 , srv1474366233:93 , srv522868692:94 , srv651781517:449 }
2023-05-22 13:53:33,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-22 13:53:33,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv941064531=98, srv1866490945=42, srv1163445478=9, srv151727304=25, srv1322605505=17, srv1203858072=12, srv1472363115=23, srv234283587=60, srv1222833994=13, srv269817899=63, srv781282988=91, srv1688921318=28, srv1146542352=8, srv1910005333=48, srv705189435=87, srv1896849683=46, srv1775862568=36, srv1888849886=44, srv1937293531=50, srv311005296=67, srv2061364137=55, srv137677414=18, srv820813025=94, srv79040743=92, srv924926808=97, srv526495924=77, srv1778046115=38, srv1011376198=1, srv571556610=80, srv1940321886=51, srv404578076=68, srv578483542=81, srv2015981010=54, srv650526611=83, srv1416347829=21, srv1820724302=40, srv1628915017=27, srv417263364=69, srv1882119651=43, srv1273404905=14, srv1753840478=34, srv1841130723=41, srv1997364286=53, srv2142906500=58, srv653551651=85, srv306879216=65, srv692427208=86, srv1199927594=11, srv1954436782=52, srv106136280=3, srv1278288115=15, srv1776884988=37, srv519207831=75, srv309357591=66, srv1073701576=4, srv858235094=95, srv1749405607=33, srv458234146=72, srv755549421=90, srv53665565=78, srv1194623599=10, srv2100350622=56, srv1767215336=35, srv1798776872=39, srv2114136515=57, srv190450371=47, srv1724972915=30, srv1747089101=32, srv130078235=16, srv544641420=79, srv1529845314=26, srv651781517=84, srv1702188711=29, srv727219709=88, srv1745437300=31, srv1107569474=5, srv418532174=70, srv522868692=76, srv1059706113=2, srv240649964=61, srv1384597321=19, srv1444276069=22, srv1007670605=0, srv509148311=74, srv138684502=20, srv1474366233=24, srv275523254=64, srv728267787=89, srv221575524=59, srv492993951=73, srv968793356=99, srv1892502629=45, srv80889046=93, srv1144262500=7, srv1126228752=6, srv599956092=82, srv437808286=71, srv873339616=96, srv1921138824=49, srv260624916=62} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-22 13:53:33,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-22 13:53:33,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-22 13:53:33,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-22 13:53:33,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-22 13:53:33,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-22 13:53:33,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-22 13:53:33,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-22 13:53:33,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-22 13:53:33,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-22 13:53:33,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-22 13:53:33,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-22 13:53:33,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-22 13:53:33,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-22 13:53:33,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-22 13:53:33,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-22 13:53:33,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-22 13:53:33,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-22 13:53:33,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-22 13:53:33,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-22 13:53:33,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-22 13:53:33,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-22 13:53:33,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-22 13:53:33,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-22 13:53:33,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-22 13:53:33,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-22 13:53:33,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-22 13:53:33,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-22 13:53:33,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-22 13:53:33,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-22 13:53:33,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-22 13:53:33,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-22 13:53:33,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-22 13:53:33,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-22 13:53:33,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-22 13:53:33,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-22 13:53:33,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-22 13:53:33,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-22 13:53:33,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-22 13:53:33,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-22 13:53:33,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-22 13:53:33,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-22 13:53:33,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-22 13:53:33,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-22 13:53:33,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-22 13:53:33,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-22 13:53:33,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-22 13:53:33,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-22 13:53:33,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-22 13:53:33,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-22 13:53:33,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-22 13:53:33,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-22 13:53:33,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-22 13:53:33,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-22 13:53:33,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-22 13:53:33,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-22 13:53:33,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-22 13:53:33,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-22 13:53:33,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-22 13:53:33,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-22 13:53:33,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-22 13:53:33,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-22 13:53:33,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-22 13:53:33,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-22 13:53:33,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-22 13:53:33,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-22 13:53:33,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-22 13:53:33,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-22 13:53:33,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-22 13:53:33,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-22 13:53:33,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-22 13:53:33,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-22 13:53:33,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-22 13:53:33,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-22 13:53:33,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-22 13:53:33,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-22 13:53:33,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-22 13:53:33,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-22 13:53:33,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-22 13:53:33,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-22 13:53:33,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-22 13:53:33,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-22 13:53:33,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-22 13:53:33,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-22 13:53:33,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-22 13:53:33,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-22 13:53:33,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-22 13:53:33,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-22 13:53:33,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-22 13:53:33,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-22 13:53:33,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-22 13:53:33,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-22 13:53:33,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-22 13:53:33,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-22 13:53:33,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-22 13:53:33,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-22 13:53:33,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-22 13:53:33,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-22 13:53:33,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-22 13:53:33,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-22 13:53:33,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-22 13:53:33,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-22 13:53:33,362 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040372069281275715, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.054933895587328876, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040372069281275715, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17541666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24083333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-22 13:53:33,362 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-22 13:53:33,362 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18004686843234102, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040372069281275715, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.054933895587328876, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040372069281275715, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17541666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24083333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-22 13:53:33,699 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 521 ms to try 5000 different iterations.  Found a solution that moves 2664 regions; Going from a computed imbalance of 0.18004686843234102 to a new imbalance of 0.01670268679411378. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021603623496951375); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03830880491781492, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.296, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021603623496951375); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013666666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04558333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-22 13:53:33,711 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1937293531:79 , srv1146542352:80 , srv1474366233:80 , srv1073701576:81 , srv2100350622:81 , srv522868692:81 , srv1126228752:82 , srv1910005333:82 , srv1921138824:82 , srv311005296:82 , srv571556610:82 , srv728267787:82 , srv106136280:83 , srv1163445478:83 , srv1194623599:83 , srv1203858072:83 , srv1702188711:83 , srv1745437300:83 , srv492993951:83 , srv79040743:83 , srv1144262500:84 , srv1278288115:84 , srv130078235:84 , srv1724972915:84 , srv1747089101:84 , srv1775862568:84 , srv1888849886:84 , srv306879216:84 , srv458234146:84 , srv705189435:84 , srv1222833994:85 , srv1273404905:85 , srv1472363115:85 , srv151727304:85 , srv1628915017:85 , srv1688921318:85 , srv1882119651:85 , srv1892502629:85 , srv1896849683:85 , srv1940321886:85 , srv1954436782:85 , srv234283587:85 , srv260624916:85 , srv275523254:85 , srv437808286:85 , srv526495924:85 , srv873339616:85 , srv1007670605:86 , srv1059706113:86 , srv1107569474:86 , srv138684502:86 , srv1444276069:86 , srv1529845314:86 , srv1997364286:86 , srv2015981010:86 , srv2061364137:86 , srv221575524:86 , srv309357591:86 , srv404578076:86 , srv417263364:86 , srv519207831:86 , srv578483542:86 , srv599956092:86 , srv692427208:86 , srv1011376198:87 , srv137677414:87 , srv1753840478:87 , srv1767215336:87 , srv1866490945:87 , srv418532174:87 , srv544641420:87 , srv653551651:87 , srv755549421:87 , srv941064531:87 , srv1199927594:88 , srv1384597321:88 , srv1416347829:88 , srv1778046115:88 , srv1820724302:88 , srv2114136515:88 , srv509148311:88 , srv727219709:88 , srv820813025:88 , srv924926808:88 , srv1322605505:89 , srv1798776872:89 , srv650526611:89 , srv968793356:89 , srv1749405607:90 , srv1776884988:90 , srv1841130723:90 , srv190450371:90 , srv2142906500:90 , srv240649964:90 , srv269817899:90 , srv53665565:90 , srv781282988:90 , srv80889046:90 , srv858235094:92 , srv651781517:517 }
2023-05-22 13:53:33,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-22 13:53:33,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv941064531=98, srv1866490945=42, srv1163445478=9, srv151727304=25, srv1322605505=17, srv1203858072=12, srv1472363115=23, srv234283587=60, srv1222833994=13, srv269817899=63, srv781282988=91, srv1688921318=28, srv1146542352=8, srv1910005333=48, srv705189435=87, srv1896849683=46, srv1775862568=36, srv1888849886=44, srv1937293531=50, srv311005296=67, srv2061364137=55, srv137677414=18, srv820813025=94, srv79040743=92, srv924926808=97, srv526495924=77, srv1778046115=38, srv1011376198=1, srv571556610=80, srv1940321886=51, srv404578076=68, srv578483542=81, srv2015981010=54, srv650526611=83, srv1416347829=21, srv1820724302=40, srv1628915017=27, srv417263364=69, srv1882119651=43, srv1273404905=14, srv1753840478=34, srv1841130723=41, srv1997364286=53, srv2142906500=58, srv653551651=85, srv306879216=65, srv692427208=86, srv1199927594=11, srv1954436782=52, srv106136280=3, srv1278288115=15, srv1776884988=37, srv519207831=75, srv309357591=66, srv1073701576=4, srv858235094=95, srv1749405607=33, srv458234146=72, srv755549421=90, srv53665565=78, srv1194623599=10, srv2100350622=56, srv1767215336=35, srv1798776872=39, srv2114136515=57, srv190450371=47, srv1724972915=30, srv1747089101=32, srv130078235=16, srv544641420=79, srv1529845314=26, srv651781517=84, srv1702188711=29, srv727219709=88, srv1745437300=31, srv1107569474=5, srv418532174=70, srv522868692=76, srv1059706113=2, srv240649964=61, srv1384597321=19, srv1444276069=22, srv1007670605=0, srv509148311=74, srv138684502=20, srv1474366233=24, srv275523254=64, srv728267787=89, srv221575524=59, srv492993951=73, srv968793356=99, srv1892502629=45, srv80889046=93, srv1144262500=7, srv1126228752=6, srv599956092=82, srv437808286=71, srv873339616=96, srv1921138824=49, srv260624916=62} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-22 13:53:33,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-22 13:53:33,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-22 13:53:33,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-22 13:53:33,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-22 13:53:33,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-22 13:53:33,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-22 13:53:33,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-22 13:53:33,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-22 13:53:33,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-22 13:53:33,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-22 13:53:33,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-22 13:53:33,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-22 13:53:33,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-22 13:53:33,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-22 13:53:33,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-22 13:53:33,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-22 13:53:33,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-22 13:53:33,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-22 13:53:33,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-22 13:53:33,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-22 13:53:33,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-22 13:53:33,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-22 13:53:33,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-22 13:53:33,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-22 13:53:33,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-22 13:53:33,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-22 13:53:33,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-22 13:53:33,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-22 13:53:33,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-22 13:53:33,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-22 13:53:33,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-22 13:53:33,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-22 13:53:33,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-22 13:53:33,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-22 13:53:33,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-22 13:53:33,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-22 13:53:33,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-22 13:53:33,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-22 13:53:33,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-22 13:53:33,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-22 13:53:33,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-22 13:53:33,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-22 13:53:33,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-22 13:53:33,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-22 13:53:33,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-22 13:53:33,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-22 13:53:33,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-22 13:53:33,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-22 13:53:33,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-22 13:53:33,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-22 13:53:33,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-22 13:53:33,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-22 13:53:33,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-22 13:53:33,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-22 13:53:33,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-22 13:53:33,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-22 13:53:33,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-22 13:53:33,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-22 13:53:33,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-22 13:53:33,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-22 13:53:33,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-22 13:53:33,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-22 13:53:33,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-22 13:53:33,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-22 13:53:33,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-22 13:53:33,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-22 13:53:33,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-22 13:53:33,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-22 13:53:33,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-22 13:53:33,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-22 13:53:33,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-22 13:53:33,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-22 13:53:33,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-22 13:53:33,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-22 13:53:33,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-22 13:53:33,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-22 13:53:33,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-22 13:53:33,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-22 13:53:33,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-22 13:53:33,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-22 13:53:33,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-22 13:53:33,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-22 13:53:33,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-22 13:53:33,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-22 13:53:33,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-22 13:53:33,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-22 13:53:33,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-22 13:53:33,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-22 13:53:33,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-22 13:53:33,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-22 13:53:33,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-22 13:53:33,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-22 13:53:33,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-22 13:53:33,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-22 13:53:33,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-22 13:53:33,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-22 13:53:33,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-22 13:53:33,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-22 13:53:33,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-22 13:53:33,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-22 13:53:33,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-22 13:53:33,904 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021603623496951375); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03830880491781492, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021603623496951375); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013666666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04558333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-22 13:53:33,904 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-22 13:53:33,904 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.016684033067419685, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021603623496951375); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03830880491781492, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021603623496951375); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013666666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04558333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-22 13:53:34,260 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 477 ms to try 5000 different iterations.  Found a solution that moves 969 regions; Going from a computed imbalance of 0.016684033067419685 to a new imbalance of 5.765874277182328E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006662924501529447); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.011121207534013126); MoveCostFunction : (multiplier=7.0, imbalance=0.10766666666666666, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006662924501529447); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00375); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-22 13:53:34,262 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv269817899:81 , srv968793356:82 , srv1749405607:83 , srv1954436782:83 , srv2142906500:83 , srv522868692:83 , srv53665565:83 , srv781282988:83 , srv858235094:83 , srv1059706113:84 , srv106136280:84 , srv1203858072:84 , srv1273404905:84 , srv1322605505:84 , srv1384597321:84 , srv1474366233:84 , srv1747089101:84 , srv1767215336:84 , srv1776884988:84 , srv1820724302:84 , srv1866490945:84 , srv1892502629:84 , srv2114136515:84 , srv417263364:84 , srv509148311:84 , srv599956092:84 , srv727219709:84 , srv79040743:84 , srv80889046:84 , srv924926808:84 , srv1194623599:85 , srv1199927594:85 , srv1416347829:85 , srv1529845314:85 , srv1628915017:85 , srv1724972915:85 , srv1841130723:85 , srv190450371:85 , srv1940321886:85 , srv240649964:85 , srv306879216:85 , srv404578076:85 , srv418532174:85 , srv437808286:85 , srv519207831:85 , srv692427208:85 , srv705189435:85 , srv755549421:85 , srv820813025:85 , srv873339616:85 , srv1007670605:86 , srv1011376198:86 , srv1073701576:86 , srv1107569474:86 , srv1126228752:86 , srv1146542352:86 , srv1222833994:86 , srv1278288115:86 , srv130078235:86 , srv137677414:86 , srv1444276069:86 , srv1688921318:86 , srv1882119651:86 , srv1896849683:86 , srv1921138824:86 , srv1937293531:86 , srv2061364137:86 , srv2100350622:86 , srv234283587:86 , srv275523254:86 , srv309357591:86 , srv311005296:86 , srv458234146:86 , srv492993951:86 , srv578483542:86 , srv650526611:86 , srv941064531:86 , srv1144262500:87 , srv138684502:87 , srv151727304:87 , srv1753840478:87 , srv1775862568:87 , srv1778046115:87 , srv1798776872:87 , srv1888849886:87 , srv1910005333:87 , srv1997364286:87 , srv2015981010:87 , srv221575524:87 , srv260624916:87 , srv526495924:87 , srv544641420:87 , srv571556610:87 , srv653551651:87 , srv728267787:87 , srv1472363115:88 , srv1702188711:88 , srv1745437300:88 , srv1163445478:89 , srv651781517:551 }
2023-05-22 13:53:34,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-22 13:53:34,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv941064531=98, srv1866490945=42, srv1163445478=9, srv151727304=25, srv1322605505=17, srv1203858072=12, srv1472363115=23, srv234283587=60, srv1222833994=13, srv269817899=63, srv781282988=91, srv1688921318=28, srv1146542352=8, srv1910005333=48, srv705189435=87, srv1896849683=46, srv1775862568=36, srv1888849886=44, srv1937293531=50, srv311005296=67, srv2061364137=55, srv137677414=18, srv820813025=94, srv79040743=92, srv924926808=97, srv526495924=77, srv1778046115=38, srv1011376198=1, srv571556610=80, srv1940321886=51, srv404578076=68, srv578483542=81, srv2015981010=54, srv650526611=83, srv1416347829=21, srv1820724302=40, srv1628915017=27, srv417263364=69, srv1882119651=43, srv1273404905=14, srv1753840478=34, srv1841130723=41, srv1997364286=53, srv2142906500=58, srv653551651=85, srv306879216=65, srv692427208=86, srv1199927594=11, srv1954436782=52, srv106136280=3, srv1278288115=15, srv1776884988=37, srv519207831=75, srv309357591=66, srv1073701576=4, srv858235094=95, srv1749405607=33, srv458234146=72, srv755549421=90, srv53665565=78, srv1194623599=10, srv2100350622=56, srv1767215336=35, srv1798776872=39, srv2114136515=57, srv190450371=47, srv1724972915=30, srv1747089101=32, srv130078235=16, srv544641420=79, srv1529845314=26, srv651781517=84, srv1702188711=29, srv727219709=88, srv1745437300=31, srv1107569474=5, srv418532174=70, srv522868692=76, srv1059706113=2, srv240649964=61, srv1384597321=19, srv1444276069=22, srv1007670605=0, srv509148311=74, srv138684502=20, srv1474366233=24, srv275523254=64, srv728267787=89, srv221575524=59, srv492993951=73, srv968793356=99, srv1892502629=45, srv80889046=93, srv1144262500=7, srv1126228752=6, srv599956092=82, srv437808286=71, srv873339616=96, srv1921138824=49, srv260624916=62} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-22 13:53:34,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-22 13:53:34,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-22 13:53:34,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-22 13:53:34,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-22 13:53:34,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-22 13:53:34,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-22 13:53:34,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-22 13:53:34,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-22 13:53:34,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-22 13:53:34,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-22 13:53:34,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-22 13:53:34,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-22 13:53:34,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-22 13:53:34,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-22 13:53:34,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-22 13:53:34,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-22 13:53:34,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-22 13:53:34,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-22 13:53:34,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-22 13:53:34,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-22 13:53:34,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-22 13:53:34,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-22 13:53:34,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-22 13:53:34,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-22 13:53:34,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-22 13:53:34,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-22 13:53:34,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-22 13:53:34,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-22 13:53:34,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-22 13:53:34,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-22 13:53:34,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-22 13:53:34,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-22 13:53:34,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-22 13:53:34,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-22 13:53:34,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-22 13:53:34,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-22 13:53:34,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-22 13:53:34,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-22 13:53:34,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-22 13:53:34,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-22 13:53:34,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-22 13:53:34,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-22 13:53:34,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-22 13:53:34,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-22 13:53:34,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-22 13:53:34,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-22 13:53:34,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-22 13:53:34,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-22 13:53:34,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-22 13:53:34,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-22 13:53:34,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-22 13:53:34,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-22 13:53:34,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-22 13:53:34,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-22 13:53:34,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-22 13:53:34,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-22 13:53:34,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-22 13:53:34,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-22 13:53:34,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-22 13:53:34,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-22 13:53:34,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-22 13:53:34,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-22 13:53:34,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-22 13:53:34,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-22 13:53:34,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-22 13:53:34,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-22 13:53:34,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-22 13:53:34,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-22 13:53:34,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-22 13:53:34,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-22 13:53:34,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-22 13:53:34,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-22 13:53:34,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-22 13:53:34,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-22 13:53:34,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-22 13:53:34,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-22 13:53:34,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-22 13:53:34,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-22 13:53:34,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-22 13:53:34,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-22 13:53:34,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-22 13:53:34,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-22 13:53:34,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-22 13:53:34,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-22 13:53:34,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-22 13:53:34,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-22 13:53:34,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-22 13:53:34,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-22 13:53:34,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-22 13:53:34,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-22 13:53:34,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-22 13:53:34,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-22 13:53:34,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-22 13:53:34,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-22 13:53:34,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-22 13:53:34,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-22 13:53:34,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-22 13:53:34,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-22 13:53:34,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-22 13:53:34,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-22 13:53:34,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-22 13:53:34,408 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006662924501529447); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.011121207534013126); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006662924501529447); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00375); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-22 13:53:34,408 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-22 13:53:34,408 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=5.698023446977456E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006662924501529447); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.011121207534013126); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006662924501529447); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00375); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-22 13:53:34,681 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 376 ms to try 5000 different iterations.  Found a solution that moves 259 regions; Going from a computed imbalance of 5.698023446977456E-4 to a new imbalance of 3.61111449348973E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.694119368783758E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.028777777777777777, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.694119368783758E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-22 13:53:34,682 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv137677414:83 , srv138684502:83 , srv1997364286:83 , srv404578076:83 , srv1007670605:84 , srv1011376198:84 , srv1073701576:84 , srv1107569474:84 , srv1163445478:84 , srv1203858072:84 , srv1384597321:84 , srv1444276069:84 , srv1472363115:84 , srv151727304:84 , srv1628915017:84 , srv1702188711:84 , srv1747089101:84 , srv1749405607:84 , srv1753840478:84 , srv1775862568:84 , srv1778046115:84 , srv1798776872:84 , srv1841130723:84 , srv1866490945:84 , srv1888849886:84 , srv1896849683:84 , srv1910005333:84 , srv2015981010:84 , srv2100350622:84 , srv2142906500:84 , srv234283587:84 , srv240649964:84 , srv309357591:84 , srv417263364:84 , srv418532174:84 , srv437808286:84 , srv458234146:84 , srv509148311:84 , srv522868692:84 , srv526495924:84 , srv544641420:84 , srv571556610:84 , srv653551651:84 , srv727219709:84 , srv728267787:84 , srv79040743:84 , srv820813025:84 , srv1059706113:85 , srv106136280:85 , srv1126228752:85 , srv1144262500:85 , srv1146542352:85 , srv1199927594:85 , srv1222833994:85 , srv1273404905:85 , srv130078235:85 , srv1322605505:85 , srv1416347829:85 , srv1474366233:85 , srv1529845314:85 , srv1688921318:85 , srv1724972915:85 , srv1745437300:85 , srv1767215336:85 , srv1776884988:85 , srv1882119651:85 , srv1892502629:85 , srv1921138824:85 , srv1937293531:85 , srv1940321886:85 , srv2061364137:85 , srv2114136515:85 , srv221575524:85 , srv260624916:85 , srv275523254:85 , srv311005296:85 , srv519207831:85 , srv53665565:85 , srv578483542:85 , srv599956092:85 , srv650526611:85 , srv755549421:85 , srv781282988:85 , srv80889046:85 , srv858235094:85 , srv873339616:85 , srv924926808:85 , srv941064531:85 , srv1194623599:86 , srv1278288115:86 , srv1820724302:86 , srv190450371:86 , srv1954436782:86 , srv269817899:86 , srv306879216:86 , srv492993951:86 , srv692427208:86 , srv705189435:86 , srv968793356:86 , srv651781517:625 }
2023-05-22 13:53:34,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-22 13:53:34,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv941064531=98, srv1866490945=42, srv1163445478=9, srv151727304=25, srv1322605505=17, srv1203858072=12, srv1472363115=23, srv234283587=60, srv1222833994=13, srv269817899=63, srv781282988=91, srv1688921318=28, srv1146542352=8, srv1910005333=48, srv705189435=87, srv1896849683=46, srv1775862568=36, srv1888849886=44, srv1937293531=50, srv311005296=67, srv2061364137=55, srv137677414=18, srv820813025=94, srv79040743=92, srv924926808=97, srv526495924=77, srv1778046115=38, srv1011376198=1, srv571556610=80, srv1940321886=51, srv404578076=68, srv578483542=81, srv2015981010=54, srv650526611=83, srv1416347829=21, srv1820724302=40, srv1628915017=27, srv417263364=69, srv1882119651=43, srv1273404905=14, srv1753840478=34, srv1841130723=41, srv1997364286=53, srv2142906500=58, srv653551651=85, srv306879216=65, srv692427208=86, srv1199927594=11, srv1954436782=52, srv106136280=3, srv1278288115=15, srv1776884988=37, srv519207831=75, srv309357591=66, srv1073701576=4, srv858235094=95, srv1749405607=33, srv458234146=72, srv755549421=90, srv53665565=78, srv1194623599=10, srv2100350622=56, srv1767215336=35, srv1798776872=39, srv2114136515=57, srv190450371=47, srv1724972915=30, srv1747089101=32, srv130078235=16, srv544641420=79, srv1529845314=26, srv651781517=84, srv1702188711=29, srv727219709=88, srv1745437300=31, srv1107569474=5, srv418532174=70, srv522868692=76, srv1059706113=2, srv240649964=61, srv1384597321=19, srv1444276069=22, srv1007670605=0, srv509148311=74, srv138684502=20, srv1474366233=24, srv275523254=64, srv728267787=89, srv221575524=59, srv492993951=73, srv968793356=99, srv1892502629=45, srv80889046=93, srv1144262500=7, srv1126228752=6, srv599956092=82, srv437808286=71, srv873339616=96, srv1921138824=49, srv260624916=62} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-22 13:53:34,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-22 13:53:34,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-22 13:53:34,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-22 13:53:34,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-22 13:53:34,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-22 13:53:34,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-22 13:53:34,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-22 13:53:34,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-22 13:53:34,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-22 13:53:34,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-22 13:53:34,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-22 13:53:34,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-22 13:53:34,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-22 13:53:34,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-22 13:53:34,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-22 13:53:34,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-22 13:53:34,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-22 13:53:34,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-22 13:53:34,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-22 13:53:34,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-22 13:53:34,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-22 13:53:34,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-22 13:53:34,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-22 13:53:34,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-22 13:53:34,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-22 13:53:34,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-22 13:53:34,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-22 13:53:34,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-22 13:53:34,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-22 13:53:34,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-22 13:53:34,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-22 13:53:34,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-22 13:53:34,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-22 13:53:34,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-22 13:53:34,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-22 13:53:34,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-22 13:53:34,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-22 13:53:34,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-22 13:53:34,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-22 13:53:34,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-22 13:53:34,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-22 13:53:34,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-22 13:53:34,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-22 13:53:34,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-22 13:53:34,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-22 13:53:34,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-22 13:53:34,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-22 13:53:34,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-22 13:53:34,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-22 13:53:34,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-22 13:53:34,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-22 13:53:34,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-22 13:53:34,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-22 13:53:34,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-22 13:53:34,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-22 13:53:34,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-22 13:53:34,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-22 13:53:34,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-22 13:53:34,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-22 13:53:34,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-22 13:53:34,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-22 13:53:34,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-22 13:53:34,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-22 13:53:34,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-22 13:53:34,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-22 13:53:34,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-22 13:53:34,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-22 13:53:34,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-22 13:53:34,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-22 13:53:34,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-22 13:53:34,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-22 13:53:34,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-22 13:53:34,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-22 13:53:34,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-22 13:53:34,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-22 13:53:34,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-22 13:53:34,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-22 13:53:34,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-22 13:53:34,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-22 13:53:34,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-22 13:53:34,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-22 13:53:34,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-22 13:53:34,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-22 13:53:34,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-22 13:53:34,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-22 13:53:34,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-22 13:53:34,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-22 13:53:34,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-22 13:53:34,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-22 13:53:34,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-22 13:53:34,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-22 13:53:34,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-22 13:53:34,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-22 13:53:34,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-22 13:53:34,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-22 13:53:34,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-22 13:53:34,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-22 13:53:34,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-22 13:53:34,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-22 13:53:34,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-22 13:53:34,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-22 13:53:34,801 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.694119368783758E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.694119368783758E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-22 13:53:34,802 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-22 13:53:34,802 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=3.4297588172971394E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.694119368783758E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.694119368783758E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-22 13:53:35,023 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 314 ms to try 5000 different iterations.  Found a solution that moves 33 regions; Going from a computed imbalance of 3.4297588172971394E-5 to a new imbalance of 6.975139992610152E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014982219165849825); MoveCostFunction : (multiplier=7.0, imbalance=0.0036666666666666666); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-22 13:53:35,024 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1194623599:83 , srv1222833994:83 , srv1529845314:83 , srv1767215336:83 , srv1776884988:83 , srv190450371:83 , srv260624916:83 , srv306879216:83 , srv578483542:83 , srv650526611:83 , srv692427208:83 , srv705189435:83 , srv941064531:83 , srv1011376198:84 , srv1059706113:84 , srv106136280:84 , srv1107569474:84 , srv1126228752:84 , srv1144262500:84 , srv1163445478:84 , srv1199927594:84 , srv1278288115:84 , srv130078235:84 , srv1322605505:84 , srv1384597321:84 , srv1416347829:84 , srv1444276069:84 , srv1472363115:84 , srv1474366233:84 , srv151727304:84 , srv1688921318:84 , srv1702188711:84 , srv1724972915:84 , srv1745437300:84 , srv1747089101:84 , srv1749405607:84 , srv1753840478:84 , srv1775862568:84 , srv1778046115:84 , srv1798776872:84 , srv1820724302:84 , srv1841130723:84 , srv1866490945:84 , srv1882119651:84 , srv1888849886:84 , srv1892502629:84 , srv1896849683:84 , srv1910005333:84 , srv1921138824:84 , srv1937293531:84 , srv1940321886:84 , srv1954436782:84 , srv1997364286:84 , srv2015981010:84 , srv2061364137:84 , srv2114136515:84 , srv2142906500:84 , srv221575524:84 , srv234283587:84 , srv240649964:84 , srv269817899:84 , srv275523254:84 , srv311005296:84 , srv404578076:84 , srv417263364:84 , srv418532174:84 , srv437808286:84 , srv492993951:84 , srv509148311:84 , srv519207831:84 , srv522868692:84 , srv53665565:84 , srv544641420:84 , srv599956092:84 , srv653551651:84 , srv727219709:84 , srv728267787:84 , srv755549421:84 , srv781282988:84 , srv79040743:84 , srv80889046:84 , srv820813025:84 , srv858235094:84 , srv873339616:84 , srv924926808:84 , srv968793356:84 , srv1007670605:85 , srv1073701576:85 , srv1146542352:85 , srv1203858072:85 , srv1273404905:85 , srv137677414:85 , srv138684502:85 , srv1628915017:85 , srv2100350622:85 , srv309357591:85 , srv458234146:85 , srv526495924:85 , srv571556610:85 , srv651781517:684 }
2023-05-22 13:53:35,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-22 13:53:35,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv941064531=98, srv1866490945=42, srv1163445478=9, srv151727304=25, srv1322605505=17, srv1203858072=12, srv1472363115=23, srv234283587=60, srv1222833994=13, srv269817899=63, srv781282988=91, srv1688921318=28, srv1146542352=8, srv1910005333=48, srv705189435=87, srv1896849683=46, srv1775862568=36, srv1888849886=44, srv1937293531=50, srv311005296=67, srv2061364137=55, srv137677414=18, srv820813025=94, srv79040743=92, srv924926808=97, srv526495924=77, srv1778046115=38, srv1011376198=1, srv571556610=80, srv1940321886=51, srv404578076=68, srv578483542=81, srv2015981010=54, srv650526611=83, srv1416347829=21, srv1820724302=40, srv1628915017=27, srv417263364=69, srv1882119651=43, srv1273404905=14, srv1753840478=34, srv1841130723=41, srv1997364286=53, srv2142906500=58, srv653551651=85, srv306879216=65, srv692427208=86, srv1199927594=11, srv1954436782=52, srv106136280=3, srv1278288115=15, srv1776884988=37, srv519207831=75, srv309357591=66, srv1073701576=4, srv858235094=95, srv1749405607=33, srv458234146=72, srv755549421=90, srv53665565=78, srv1194623599=10, srv2100350622=56, srv1767215336=35, srv1798776872=39, srv2114136515=57, srv190450371=47, srv1724972915=30, srv1747089101=32, srv130078235=16, srv544641420=79, srv1529845314=26, srv651781517=84, srv1702188711=29, srv727219709=88, srv1745437300=31, srv1107569474=5, srv418532174=70, srv522868692=76, srv1059706113=2, srv240649964=61, srv1384597321=19, srv1444276069=22, srv1007670605=0, srv509148311=74, srv138684502=20, srv1474366233=24, srv275523254=64, srv728267787=89, srv221575524=59, srv492993951=73, srv968793356=99, srv1892502629=45, srv80889046=93, srv1144262500=7, srv1126228752=6, srv599956092=82, srv437808286=71, srv873339616=96, srv1921138824=49, srv260624916=62} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-22 13:53:35,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-22 13:53:35,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-22 13:53:35,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-22 13:53:35,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-22 13:53:35,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-22 13:53:35,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-22 13:53:35,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-22 13:53:35,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-22 13:53:35,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-22 13:53:35,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-22 13:53:35,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-22 13:53:35,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-22 13:53:35,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-22 13:53:35,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-22 13:53:35,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-22 13:53:35,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-22 13:53:35,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-22 13:53:35,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-22 13:53:35,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-22 13:53:35,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-22 13:53:35,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-22 13:53:35,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-22 13:53:35,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-22 13:53:35,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-22 13:53:35,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-22 13:53:35,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-22 13:53:35,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-22 13:53:35,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-22 13:53:35,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-22 13:53:35,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-22 13:53:35,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-22 13:53:35,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-22 13:53:35,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-22 13:53:35,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-22 13:53:35,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-22 13:53:35,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-22 13:53:35,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-22 13:53:35,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-22 13:53:35,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-22 13:53:35,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-22 13:53:35,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-22 13:53:35,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-22 13:53:35,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-22 13:53:35,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-22 13:53:35,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-22 13:53:35,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-22 13:53:35,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-22 13:53:35,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-22 13:53:35,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-22 13:53:35,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-22 13:53:35,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-22 13:53:35,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-22 13:53:35,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-22 13:53:35,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-22 13:53:35,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-22 13:53:35,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-22 13:53:35,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-22 13:53:35,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-22 13:53:35,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-22 13:53:35,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-22 13:53:35,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-22 13:53:35,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-22 13:53:35,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-22 13:53:35,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-22 13:53:35,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-22 13:53:35,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-22 13:53:35,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-22 13:53:35,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-22 13:53:35,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-22 13:53:35,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-22 13:53:35,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-22 13:53:35,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-22 13:53:35,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-22 13:53:35,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-22 13:53:35,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-22 13:53:35,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-22 13:53:35,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-22 13:53:35,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-22 13:53:35,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-22 13:53:35,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-22 13:53:35,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-22 13:53:35,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-22 13:53:35,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-22 13:53:35,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-22 13:53:35,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-22 13:53:35,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-22 13:53:35,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-22 13:53:35,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-22 13:53:35,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-22 13:53:35,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-22 13:53:35,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-22 13:53:35,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-22 13:53:35,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-22 13:53:35,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-22 13:53:35,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-22 13:53:35,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-22 13:53:35,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-22 13:53:35,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-22 13:53:35,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-22 13:53:35,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-22 13:53:35,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-22 13:53:35,151 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=6.744069053831948E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014982219165849825); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-22 13:53:35,152 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1194623599:83 , srv1222833994:83 , srv1529845314:83 , srv1767215336:83 , srv1776884988:83 , srv190450371:83 , srv260624916:83 , srv306879216:83 , srv578483542:83 , srv650526611:83 , srv692427208:83 , srv705189435:83 , srv941064531:83 , srv1011376198:84 , srv1059706113:84 , srv106136280:84 , srv1107569474:84 , srv1126228752:84 , srv1144262500:84 , srv1163445478:84 , srv1199927594:84 , srv1278288115:84 , srv130078235:84 , srv1322605505:84 , srv1384597321:84 , srv1416347829:84 , srv1444276069:84 , srv1472363115:84 , srv1474366233:84 , srv151727304:84 , srv1688921318:84 , srv1702188711:84 , srv1724972915:84 , srv1745437300:84 , srv1747089101:84 , srv1749405607:84 , srv1753840478:84 , srv1775862568:84 , srv1778046115:84 , srv1798776872:84 , srv1820724302:84 , srv1841130723:84 , srv1866490945:84 , srv1882119651:84 , srv1888849886:84 , srv1892502629:84 , srv1896849683:84 , srv1910005333:84 , srv1921138824:84 , srv1937293531:84 , srv1940321886:84 , srv1954436782:84 , srv1997364286:84 , srv2015981010:84 , srv2061364137:84 , srv2114136515:84 , srv2142906500:84 , srv221575524:84 , srv234283587:84 , srv240649964:84 , srv269817899:84 , srv275523254:84 , srv311005296:84 , srv404578076:84 , srv417263364:84 , srv418532174:84 , srv437808286:84 , srv492993951:84 , srv509148311:84 , srv519207831:84 , srv522868692:84 , srv53665565:84 , srv544641420:84 , srv599956092:84 , srv653551651:84 , srv727219709:84 , srv728267787:84 , srv755549421:84 , srv781282988:84 , srv79040743:84 , srv80889046:84 , srv820813025:84 , srv858235094:84 , srv873339616:84 , srv924926808:84 , srv968793356:84 , srv1007670605:85 , srv1073701576:85 , srv1146542352:85 , srv1203858072:85 , srv1273404905:85 , srv137677414:85 , srv138684502:85 , srv1628915017:85 , srv2100350622:85 , srv309357591:85 , srv458234146:85 , srv526495924:85 , srv571556610:85 , srv651781517:684 }
2023-05-22 13:53:35,178 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=264 (was 264), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=312 (was 321), ProcessCount=171 (was 171), AvailableMemoryMB=6718 (was 7343)
2023-05-22 13:53:35,185 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=264, MaxFileDescriptor=60000, SystemLoadAverage=312, ProcessCount=171, AvailableMemoryMB=6717
2023-05-22 13:53:35,186 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-22 13:53:35,186 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-05-22 13:53:35,198 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1040545368:84 , srv1062329757:84 , srv1063448438:84 , srv1097824443:84 , srv1111199230:84 , srv1125471077:84 , srv1130322127:84 , srv113055235:84 , srv115128424:84 , srv1204625907:84 , srv1267136247:84 , srv1350762095:84 , srv1506856654:84 , srv1564971538:84 , srv1833057357:84 , srv1903913917:84 , srv1922133086:84 , srv2074718763:84 , srv2081025474:84 , srv2136089424:84 , srv388449624:84 , srv469252681:84 , srv578987359:84 , srv646390160:84 , srv653673484:84 , srv86213427:84 , srv893594606:84 , srv918943778:84 , srv945024923:84 , srv941545135:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-05-22 13:53:35,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-22 13:53:35,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1564971538=13, srv918943778=27, srv893594606=26, srv578987359=22, srv1040545368=0, srv1111199230=4, srv1506856654=12, srv2081025474=18, srv388449624=20, srv115128424=8, srv1204625907=9, srv941545135=28, srv1833057357=14, srv945024923=29, srv1125471077=5, srv1903913917=15, srv469252681=21, srv1063448438=2, srv2136089424=19, srv1130322127=6, srv1097824443=3, srv1350762095=11, srv1062329757=1, srv86213427=25, srv2074718763=17, srv1267136247=10, srv653673484=24, srv113055235=7, srv1922133086=16, srv646390160=23} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-22 13:53:35,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-22 13:53:35,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-22 13:53:35,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-22 13:53:35,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-22 13:53:35,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-22 13:53:35,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-22 13:53:35,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-22 13:53:35,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-22 13:53:35,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-22 13:53:35,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-22 13:53:35,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-22 13:53:35,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-22 13:53:35,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-22 13:53:35,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-22 13:53:35,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-22 13:53:35,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-22 13:53:35,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-22 13:53:35,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-22 13:53:35,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-22 13:53:35,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-22 13:53:35,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-22 13:53:35,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-22 13:53:35,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-22 13:53:35,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-22 13:53:35,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-22 13:53:35,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-22 13:53:35,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-22 13:53:35,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-22 13:53:35,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-22 13:53:35,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-22 13:53:35,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-05-22 13:53:35,239 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1773426916677994, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-22 13:53:35,240 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912751844294951, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1773426916677994, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-05-22 13:55:35,219 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6561048 different iterations.  Found a solution that moves 1848 regions; Going from a computed imbalance of 0.9912751844294951 to a new imbalance of 4.313324190526491E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6844444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-22 13:55:35,226 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1040545368:90 , srv1062329757:90 , srv1063448438:90 , srv1097824443:90 , srv1111199230:90 , srv1125471077:90 , srv1130322127:90 , srv113055235:90 , srv115128424:90 , srv1204625907:90 , srv1267136247:90 , srv1350762095:90 , srv1506856654:90 , srv1564971538:90 , srv1833057357:90 , srv1903913917:90 , srv1922133086:90 , srv2074718763:90 , srv2081025474:90 , srv2136089424:90 , srv388449624:90 , srv469252681:90 , srv578987359:90 , srv646390160:90 , srv653673484:90 , srv86213427:90 , srv893594606:90 , srv918943778:90 , srv941545135:90 , srv945024923:90 }
2023-05-22 13:55:35,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-22 13:55:35,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1564971538=13, srv918943778=27, srv893594606=26, srv578987359=22, srv1040545368=0, srv1111199230=4, srv1506856654=12, srv2081025474=18, srv388449624=20, srv115128424=8, srv1204625907=9, srv941545135=28, srv1833057357=14, srv945024923=29, srv1125471077=5, srv1903913917=15, srv469252681=21, srv1063448438=2, srv2136089424=19, srv1130322127=6, srv1097824443=3, srv1350762095=11, srv1062329757=1, srv86213427=25, srv2074718763=17, srv1267136247=10, srv653673484=24, srv113055235=7, srv1922133086=16, srv646390160=23} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-22 13:55:35,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-22 13:55:35,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-22 13:55:35,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-22 13:55:35,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-22 13:55:35,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-22 13:55:35,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-22 13:55:35,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-22 13:55:35,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-22 13:55:35,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-22 13:55:35,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-22 13:55:35,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-22 13:55:35,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-22 13:55:35,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-22 13:55:35,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-22 13:55:35,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-22 13:55:35,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-22 13:55:35,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-22 13:55:35,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-22 13:55:35,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-22 13:55:35,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-22 13:55:35,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-22 13:55:35,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-22 13:55:35,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-22 13:55:35,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-22 13:55:35,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-22 13:55:35,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-22 13:55:35,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-22 13:55:35,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-22 13:55:35,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-22 13:55:35,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-22 13:55:35,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-05-22 13:55:35,251 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-22 13:55:35,251 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1040545368:90 , srv1062329757:90 , srv1063448438:90 , srv1097824443:90 , srv1111199230:90 , srv1125471077:90 , srv1130322127:90 , srv113055235:90 , srv115128424:90 , srv1204625907:90 , srv1267136247:90 , srv1350762095:90 , srv1506856654:90 , srv1564971538:90 , srv1833057357:90 , srv1903913917:90 , srv1922133086:90 , srv2074718763:90 , srv2081025474:90 , srv2136089424:90 , srv388449624:90 , srv469252681:90 , srv578987359:90 , srv646390160:90 , srv653673484:90 , srv86213427:90 , srv893594606:90 , srv918943778:90 , srv941545135:90 , srv945024923:90 }
2023-05-22 13:55:35,270 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=264 (was 264), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=198 (was 312), ProcessCount=169 (was 171), AvailableMemoryMB=6421 (was 6717)