2023-02-17 14:54:01,506 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-02-17 14:54:01,843 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-02-17 14:54:01,964 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-02-17 14:54:02,029 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-17 14:54:02,036 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-02-17 14:54:02,049 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=270, ProcessCount=170, AvailableMemoryMB=8849
2023-02-17 14:54:02,051 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-17 14:54:02,051 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-02-17 14:54:02,197 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1036204398:84 , srv1042970641:84 , srv1052609107:84 , srv1069096274:84 , srv1074889415:84 , srv1076178973:84 , srv1105262010:84 , srv1127988790:84 , srv113156719:84 , srv1151785161:84 , srv1157791936:84 , srv1158272895:84 , srv1176540314:84 , srv1179022036:84 , srv1202193951:84 , srv1219541171:84 , srv1247857880:84 , srv125528966:84 , srv1271006086:84 , srv127637808:84 , srv1283819826:84 , srv1295223106:84 , srv1297753351:84 , srv1304085989:84 , srv1305015618:84 , srv1316181984:84 , srv1372701682:84 , srv1395793787:84 , srv1407371508:84 , srv1453080029:84 , srv1454258926:84 , srv1455710708:84 , srv1455984182:84 , srv1457515110:84 , srv1480093961:84 , srv1492911567:84 , srv1539783001:84 , srv1591503979:84 , srv1594322194:84 , srv1598000879:84 , srv162279861:84 , srv1649424345:84 , srv1668332594:84 , srv1714728566:84 , srv1757631410:84 , srv1770602525:84 , srv1810191151:84 , srv1844084434:84 , srv1852635107:84 , srv1858339455:84 , srv1874773592:84 , srv1899385864:84 , srv1911205155:84 , srv1912592226:84 , srv1914794846:84 , srv1925625083:84 , srv1939831338:84 , srv1992955522:84 , srv2011531881:84 , srv2064962851:84 , srv2089518820:84 , srv209142571:84 , srv2103112576:84 , srv2111623629:84 , srv2114362880:84 , srv2125691280:84 , srv2130394908:84 , srv2145183053:84 , srv244067715:84 , srv31048356:84 , srv319213505:84 , srv328496183:84 , srv353690749:84 , srv357776541:84 , srv37085901:84 , srv379381707:84 , srv392246547:84 , srv4452896:84 , srv470944088:84 , srv477467692:84 , srv496832077:84 , srv499324579:84 , srv520737830:84 , srv534413083:84 , srv53779164:84 , srv56022996:84 , srv562786161:84 , srv583746224:84 , srv588476118:84 , srv627834816:84 , srv65746255:84 , srv690078825:84 , srv712386493:84 , srv731852234:84 , srv807842089:84 , srv926962561:84 , srv953012760:84 , srv975320895:84 , srv982488167:84 , srv2058907738:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-02-17 14:54:02,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-17 14:54:02,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1899385864=51, srv1539783001=36, srv534413083=84, srv807842089=95, srv1271006086=18, srv1305015618=24, srv1316181984=25, srv1852635107=48, srv1219541171=15, srv496832077=81, srv1911205155=52, srv627834816=90, srv1453080029=29, srv520737830=83, srv975320895=98, srv1247857880=16, srv1042970641=1, srv1372701682=26, srv1304085989=23, srv1179022036=13, srv125528966=17, srv2114362880=65, srv1668332594=42, srv162279861=40, srv2058907738=59, srv1858339455=49, srv1076178973=5, srv2125691280=66, srv1395793787=27, srv1202193951=14, srv1036204398=0, srv1157791936=10, srv1295223106=21, srv1455710708=31, srv1283819826=20, srv588476118=89, srv1770602525=45, srv470944088=79, srv953012760=97, srv37085901=75, srv319213505=71, srv1105262010=6, srv1757631410=44, srv982488167=99, srv1457515110=33, srv31048356=70, srv1912592226=53, srv2011531881=58, srv4452896=78, srv1939831338=56, srv690078825=92, srv1914794846=54, srv209142571=62, srv1454258926=30, srv127637808=19, srv499324579=82, srv1297753351=22, srv2064962851=60, srv56022996=86, srv1158272895=11, srv1407371508=28, srv1480093961=34, srv1598000879=39, srv2145183053=68, srv1176540314=12, srv1810191151=46, srv1925625083=55, srv1649424345=41, srv1591503979=37, srv357776541=74, srv328496183=72, srv731852234=94, srv583746224=88, srv1714728566=43, srv353690749=73, srv113156719=8, srv1992955522=57, srv477467692=80, srv392246547=77, srv1127988790=7, srv53779164=85, srv2111623629=64, srv1455984182=32, srv2089518820=61, srv1844084434=47, srv712386493=93, srv1052609107=2, srv1594322194=38, srv1492911567=35, srv1874773592=50, srv1069096274=3, srv65746255=91, srv926962561=96, srv1074889415=4, srv2130394908=67, srv562786161=87, srv379381707=76, srv1151785161=9, srv2103112576=63, srv244067715=69} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-17 14:54:02,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-17 14:54:02,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-17 14:54:02,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-17 14:54:02,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-17 14:54:02,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-17 14:54:02,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-17 14:54:02,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-17 14:54:02,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-17 14:54:02,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-17 14:54:02,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-17 14:54:02,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-17 14:54:02,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-17 14:54:02,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-17 14:54:02,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-17 14:54:02,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-17 14:54:02,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-17 14:54:02,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-17 14:54:02,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-17 14:54:02,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-17 14:54:02,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-17 14:54:02,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-17 14:54:02,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-17 14:54:02,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-17 14:54:02,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-17 14:54:02,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-17 14:54:02,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-17 14:54:02,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-17 14:54:02,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-17 14:54:02,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-17 14:54:02,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-17 14:54:02,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-17 14:54:02,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-17 14:54:02,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-17 14:54:02,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-17 14:54:02,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-17 14:54:02,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-17 14:54:02,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-17 14:54:02,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-17 14:54:02,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-17 14:54:02,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-17 14:54:02,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-17 14:54:02,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-17 14:54:02,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-17 14:54:02,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-17 14:54:02,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-17 14:54:02,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-17 14:54:02,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-17 14:54:02,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-17 14:54:02,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-17 14:54:02,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-17 14:54:02,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-17 14:54:02,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-17 14:54:02,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-17 14:54:02,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-17 14:54:02,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-17 14:54:02,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-17 14:54:02,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-17 14:54:02,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-17 14:54:02,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-17 14:54:02,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-17 14:54:02,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-17 14:54:02,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-17 14:54:02,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-17 14:54:02,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-17 14:54:02,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-17 14:54:02,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-17 14:54:02,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-17 14:54:02,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-17 14:54:02,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-17 14:54:02,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-17 14:54:02,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-17 14:54:02,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-17 14:54:02,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-17 14:54:02,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-17 14:54:02,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-17 14:54:02,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-17 14:54:02,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-17 14:54:02,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-17 14:54:02,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-17 14:54:02,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-17 14:54:02,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-17 14:54:02,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-17 14:54:02,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-17 14:54:02,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-17 14:54:02,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-17 14:54:02,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-17 14:54:02,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-17 14:54:02,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-17 14:54:02,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-17 14:54:02,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-17 14:54:02,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-17 14:54:02,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-17 14:54:02,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-17 14:54:02,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-17 14:54:02,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-17 14:54:02,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-17 14:54:02,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-17 14:54:02,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-17 14:54:02,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-17 14:54:02,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-17 14:54:02,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-17 14:54:02,594 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-17 14:54:02,594 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-17 14:54:02,594 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-17 14:54:02,915 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 564 ms to try 5000 different iterations.  Found a solution that moves 4747 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.16813546085050302. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.042657687046096494, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0522705557544366, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5274444444444445, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.042657687046096494, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.163, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23233333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-17 14:54:02,933 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1042970641:82 , srv113156719:82 , srv1271006086:82 , srv1283819826:82 , srv1052609107:83 , srv1127988790:83 , srv1157791936:83 , srv1179022036:83 , srv1454258926:83 , srv1455984182:83 , srv162279861:83 , srv1649424345:83 , srv1757631410:83 , srv1911205155:83 , srv1914794846:83 , srv328496183:83 , srv499324579:83 , srv56022996:83 , srv1069096274:84 , srv1105262010:84 , srv1219541171:84 , srv1297753351:84 , srv1395793787:84 , srv1539783001:84 , srv1591503979:84 , srv1598000879:84 , srv1668332594:84 , srv1810191151:84 , srv1874773592:84 , srv2130394908:84 , srv690078825:84 , srv731852234:84 , srv1176540314:85 , srv1202193951:85 , srv1304085989:85 , srv1457515110:85 , srv1714728566:85 , srv1912592226:85 , srv2125691280:85 , srv244067715:85 , srv31048356:85 , srv37085901:85 , srv982488167:85 , srv1036204398:86 , srv1074889415:86 , srv1151785161:86 , srv1158272895:86 , srv1295223106:86 , srv1594322194:86 , srv1844084434:86 , srv2064962851:86 , srv2089518820:86 , srv209142571:86 , srv379381707:86 , srv392246547:86 , srv4452896:86 , srv588476118:86 , srv65746255:86 , srv712386493:86 , srv926962561:86 , srv953012760:86 , srv1247857880:87 , srv1407371508:87 , srv1899385864:87 , srv1925625083:87 , srv2114362880:87 , srv353690749:87 , srv477467692:87 , srv562786161:87 , srv807842089:87 , srv1076178973:88 , srv125528966:88 , srv1305015618:88 , srv1316181984:88 , srv1372701682:88 , srv1480093961:88 , srv1492911567:88 , srv1858339455:88 , srv1939831338:88 , srv2103112576:88 , srv2111623629:88 , srv319213505:88 , srv520737830:88 , srv627834816:88 , srv1852635107:89 , srv1992955522:89 , srv2145183053:89 , srv357776541:89 , srv470944088:89 , srv534413083:89 , srv53779164:89 , srv583746224:89 , srv2011531881:90 , srv1770602525:91 , srv496832077:93 , srv1455710708:94 , srv127637808:95 , srv1453080029:95 , srv975320895:97 , srv2058907738:469 }
2023-02-17 14:54:02,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-17 14:54:02,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1899385864=51, srv1539783001=36, srv534413083=84, srv807842089=95, srv1271006086=18, srv1305015618=24, srv1316181984=25, srv1852635107=48, srv1219541171=15, srv496832077=81, srv1911205155=52, srv627834816=90, srv1453080029=29, srv520737830=83, srv975320895=98, srv1247857880=16, srv1042970641=1, srv1372701682=26, srv1304085989=23, srv1179022036=13, srv125528966=17, srv2114362880=65, srv1668332594=42, srv162279861=40, srv2058907738=59, srv1858339455=49, srv1076178973=5, srv2125691280=66, srv1395793787=27, srv1202193951=14, srv1036204398=0, srv1157791936=10, srv1295223106=21, srv1455710708=31, srv1283819826=20, srv588476118=89, srv1770602525=45, srv470944088=79, srv953012760=97, srv37085901=75, srv319213505=71, srv1105262010=6, srv1757631410=44, srv982488167=99, srv1457515110=33, srv31048356=70, srv1912592226=53, srv2011531881=58, srv4452896=78, srv1939831338=56, srv690078825=92, srv1914794846=54, srv209142571=62, srv1454258926=30, srv127637808=19, srv499324579=82, srv1297753351=22, srv2064962851=60, srv56022996=86, srv1158272895=11, srv1407371508=28, srv1480093961=34, srv1598000879=39, srv2145183053=68, srv1176540314=12, srv1810191151=46, srv1925625083=55, srv1649424345=41, srv1591503979=37, srv357776541=74, srv328496183=72, srv731852234=94, srv583746224=88, srv1714728566=43, srv353690749=73, srv113156719=8, srv1992955522=57, srv477467692=80, srv392246547=77, srv1127988790=7, srv53779164=85, srv2111623629=64, srv1455984182=32, srv2089518820=61, srv1844084434=47, srv712386493=93, srv1052609107=2, srv1594322194=38, srv1492911567=35, srv1874773592=50, srv1069096274=3, srv65746255=91, srv926962561=96, srv1074889415=4, srv2130394908=67, srv562786161=87, srv379381707=76, srv1151785161=9, srv2103112576=63, srv244067715=69} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-17 14:54:02,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-17 14:54:02,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-17 14:54:02,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-17 14:54:02,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-17 14:54:02,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-17 14:54:02,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-17 14:54:02,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-17 14:54:02,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-17 14:54:02,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-17 14:54:02,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-17 14:54:02,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-17 14:54:02,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-17 14:54:02,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-17 14:54:02,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-17 14:54:02,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-17 14:54:02,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-17 14:54:02,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-17 14:54:02,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-17 14:54:02,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-17 14:54:02,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-17 14:54:02,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-17 14:54:02,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-17 14:54:02,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-17 14:54:02,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-17 14:54:02,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-17 14:54:02,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-17 14:54:02,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-17 14:54:02,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-17 14:54:02,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-17 14:54:02,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-17 14:54:02,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-17 14:54:02,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-17 14:54:02,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-17 14:54:02,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-17 14:54:02,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-17 14:54:02,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-17 14:54:02,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-17 14:54:02,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-17 14:54:02,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-17 14:54:02,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-17 14:54:02,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-17 14:54:02,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-17 14:54:02,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-17 14:54:02,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-17 14:54:02,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-17 14:54:02,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-17 14:54:02,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-17 14:54:02,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-17 14:54:02,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-17 14:54:02,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-17 14:54:02,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-17 14:54:02,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-17 14:54:02,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-17 14:54:02,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-17 14:54:02,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-17 14:54:02,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-17 14:54:02,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-17 14:54:02,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-17 14:54:02,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-17 14:54:02,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-17 14:54:02,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-17 14:54:02,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-17 14:54:02,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-17 14:54:02,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-17 14:54:02,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-17 14:54:02,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-17 14:54:02,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-17 14:54:02,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-17 14:54:02,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-17 14:54:02,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-17 14:54:02,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-17 14:54:02,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-17 14:54:02,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-17 14:54:02,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-17 14:54:02,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-17 14:54:02,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-17 14:54:02,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-17 14:54:02,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-17 14:54:02,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-17 14:54:02,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-17 14:54:02,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-17 14:54:02,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-17 14:54:02,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-17 14:54:02,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-17 14:54:02,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-17 14:54:02,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-17 14:54:02,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-17 14:54:02,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-17 14:54:02,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-17 14:54:02,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-17 14:54:02,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-17 14:54:02,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-17 14:54:02,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-17 14:54:02,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-17 14:54:02,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-17 14:54:02,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-17 14:54:02,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-17 14:54:02,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-17 14:54:02,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-17 14:54:02,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-17 14:54:02,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-17 14:54:03,112 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.042657687046096494, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0522705557544366, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.042657687046096494, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.163, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23233333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-17 14:54:03,112 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-17 14:54:03,112 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.16810222164606725, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.042657687046096494, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0522705557544366, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.042657687046096494, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.163, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23233333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-17 14:54:03,399 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 402 ms to try 5000 different iterations.  Found a solution that moves 2585 regions; Going from a computed imbalance of 0.16810222164606725 to a new imbalance of 0.01600595078395957. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02319369256898116); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03194762829983295, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2872222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02319369256898116); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04475, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-17 14:54:03,408 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv975320895:75 , srv1453080029:76 , srv127637808:78 , srv496832077:79 , srv1939831338:81 , srv1305015618:82 , srv1770602525:82 , srv357776541:82 , srv520737830:82 , srv1151785161:83 , srv125528966:83 , srv1316181984:83 , srv1480093961:83 , srv1492911567:83 , srv2011531881:83 , srv353690749:83 , srv470944088:83 , srv534413083:83 , srv1076178973:84 , srv1372701682:84 , srv1844084434:84 , srv1852635107:84 , srv1899385864:84 , srv1925625083:84 , srv2111623629:84 , srv2125691280:84 , srv65746255:84 , srv807842089:84 , srv1295223106:85 , srv1455710708:85 , srv1992955522:85 , srv209142571:85 , srv2114362880:85 , srv2145183053:85 , srv319213505:85 , srv477467692:85 , srv53779164:85 , srv731852234:85 , srv926962561:85 , srv982488167:85 , srv1074889415:86 , srv1176540314:86 , srv1202193951:86 , srv1247857880:86 , srv1304085989:86 , srv1454258926:86 , srv1457515110:86 , srv1594322194:86 , srv1598000879:86 , srv1668332594:86 , srv1757631410:86 , srv1858339455:86 , srv2103112576:86 , srv37085901:86 , srv379381707:86 , srv4452896:86 , srv588476118:86 , srv1158272895:87 , srv1395793787:87 , srv1714728566:87 , srv1874773592:87 , srv1912592226:87 , srv2064962851:87 , srv2089518820:87 , srv2130394908:87 , srv31048356:87 , srv562786161:87 , srv583746224:87 , srv690078825:87 , srv712386493:87 , srv1052609107:88 , srv1127988790:88 , srv1157791936:88 , srv1179022036:88 , srv1219541171:88 , srv1407371508:88 , srv1455984182:88 , srv1591503979:88 , srv1914794846:88 , srv392246547:88 , srv499324579:88 , srv56022996:88 , srv627834816:88 , srv1036204398:89 , srv1069096274:89 , srv1105262010:89 , srv113156719:89 , srv1283819826:89 , srv1297753351:89 , srv1539783001:89 , srv162279861:89 , srv1649424345:89 , srv1810191151:89 , srv1911205155:89 , srv244067715:89 , srv328496183:89 , srv953012760:89 , srv1042970641:91 , srv1271006086:91 , srv2058907738:511 }
2023-02-17 14:54:03,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-17 14:54:03,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1899385864=51, srv1539783001=36, srv534413083=84, srv807842089=95, srv1271006086=18, srv1305015618=24, srv1316181984=25, srv1852635107=48, srv1219541171=15, srv496832077=81, srv1911205155=52, srv627834816=90, srv1453080029=29, srv520737830=83, srv975320895=98, srv1247857880=16, srv1042970641=1, srv1372701682=26, srv1304085989=23, srv1179022036=13, srv125528966=17, srv2114362880=65, srv1668332594=42, srv162279861=40, srv2058907738=59, srv1858339455=49, srv1076178973=5, srv2125691280=66, srv1395793787=27, srv1202193951=14, srv1036204398=0, srv1157791936=10, srv1295223106=21, srv1455710708=31, srv1283819826=20, srv588476118=89, srv1770602525=45, srv470944088=79, srv953012760=97, srv37085901=75, srv319213505=71, srv1105262010=6, srv1757631410=44, srv982488167=99, srv1457515110=33, srv31048356=70, srv1912592226=53, srv2011531881=58, srv4452896=78, srv1939831338=56, srv690078825=92, srv1914794846=54, srv209142571=62, srv1454258926=30, srv127637808=19, srv499324579=82, srv1297753351=22, srv2064962851=60, srv56022996=86, srv1158272895=11, srv1407371508=28, srv1480093961=34, srv1598000879=39, srv2145183053=68, srv1176540314=12, srv1810191151=46, srv1925625083=55, srv1649424345=41, srv1591503979=37, srv357776541=74, srv328496183=72, srv731852234=94, srv583746224=88, srv1714728566=43, srv353690749=73, srv113156719=8, srv1992955522=57, srv477467692=80, srv392246547=77, srv1127988790=7, srv53779164=85, srv2111623629=64, srv1455984182=32, srv2089518820=61, srv1844084434=47, srv712386493=93, srv1052609107=2, srv1594322194=38, srv1492911567=35, srv1874773592=50, srv1069096274=3, srv65746255=91, srv926962561=96, srv1074889415=4, srv2130394908=67, srv562786161=87, srv379381707=76, srv1151785161=9, srv2103112576=63, srv244067715=69} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-17 14:54:03,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-17 14:54:03,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-17 14:54:03,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-17 14:54:03,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-17 14:54:03,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-17 14:54:03,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-17 14:54:03,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-17 14:54:03,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-17 14:54:03,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-17 14:54:03,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-17 14:54:03,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-17 14:54:03,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-17 14:54:03,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-17 14:54:03,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-17 14:54:03,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-17 14:54:03,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-17 14:54:03,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-17 14:54:03,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-17 14:54:03,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-17 14:54:03,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-17 14:54:03,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-17 14:54:03,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-17 14:54:03,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-17 14:54:03,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-17 14:54:03,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-17 14:54:03,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-17 14:54:03,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-17 14:54:03,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-17 14:54:03,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-17 14:54:03,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-17 14:54:03,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-17 14:54:03,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-17 14:54:03,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-17 14:54:03,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-17 14:54:03,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-17 14:54:03,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-17 14:54:03,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-17 14:54:03,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-17 14:54:03,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-17 14:54:03,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-17 14:54:03,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-17 14:54:03,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-17 14:54:03,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-17 14:54:03,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-17 14:54:03,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-17 14:54:03,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-17 14:54:03,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-17 14:54:03,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-17 14:54:03,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-17 14:54:03,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-17 14:54:03,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-17 14:54:03,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-17 14:54:03,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-17 14:54:03,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-17 14:54:03,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-17 14:54:03,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-17 14:54:03,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-17 14:54:03,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-17 14:54:03,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-17 14:54:03,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-17 14:54:03,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-17 14:54:03,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-17 14:54:03,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-17 14:54:03,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-17 14:54:03,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-17 14:54:03,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-17 14:54:03,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-17 14:54:03,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-17 14:54:03,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-17 14:54:03,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-17 14:54:03,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-17 14:54:03,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-17 14:54:03,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-17 14:54:03,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-17 14:54:03,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-17 14:54:03,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-17 14:54:03,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-17 14:54:03,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-17 14:54:03,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-17 14:54:03,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-17 14:54:03,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-17 14:54:03,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-17 14:54:03,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-17 14:54:03,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-17 14:54:03,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-17 14:54:03,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-17 14:54:03,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-17 14:54:03,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-17 14:54:03,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-17 14:54:03,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-17 14:54:03,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-17 14:54:03,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-17 14:54:03,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-17 14:54:03,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-17 14:54:03,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-17 14:54:03,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-17 14:54:03,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-17 14:54:03,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-17 14:54:03,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-17 14:54:03,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-17 14:54:03,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-17 14:54:03,574 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02319369256898116); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03194762829983295, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02319369256898116); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04475, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-17 14:54:03,574 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-17 14:54:03,574 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.01598785022708861, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02319369256898116); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03194762829983295, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02319369256898116); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04475, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-17 14:54:03,812 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 332 ms to try 5000 different iterations.  Found a solution that moves 985 regions; Going from a computed imbalance of 0.01598785022708861 to a new imbalance of 5.9961380699849E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005885806956706312); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00704323668071124); MoveCostFunction : (multiplier=7.0, imbalance=0.10944444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005885806956706312); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-17 14:54:03,821 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1455710708:78 , srv583746224:82 , srv627834816:82 , srv953012760:82 , srv1158272895:83 , srv1271006086:83 , srv1407371508:83 , srv2103112576:83 , srv392246547:83 , srv53779164:83 , srv1036204398:84 , srv1069096274:84 , srv1297753351:84 , srv1539783001:84 , srv1858339455:84 , srv1992955522:84 , srv2089518820:84 , srv2145183053:84 , srv244067715:84 , srv319213505:84 , srv562786161:84 , srv1042970641:85 , srv1074889415:85 , srv1076178973:85 , srv1105262010:85 , srv1219541171:85 , srv1247857880:85 , srv127637808:85 , srv1591503979:85 , srv1594322194:85 , srv162279861:85 , srv1714728566:85 , srv1770602525:85 , srv1911205155:85 , srv2011531881:85 , srv2064962851:85 , srv31048356:85 , srv379381707:85 , srv470944088:85 , srv477467692:85 , srv712386493:85 , srv807842089:85 , srv975320895:85 , srv1127988790:86 , srv1157791936:86 , srv1176540314:86 , srv1179022036:86 , srv1202193951:86 , srv1283819826:86 , srv1304085989:86 , srv1305015618:86 , srv1316181984:86 , srv1372701682:86 , srv1395793787:86 , srv1453080029:86 , srv1455984182:86 , srv1810191151:86 , srv1852635107:86 , srv1874773592:86 , srv1899385864:86 , srv1912592226:86 , srv2111623629:86 , srv2130394908:86 , srv328496183:86 , srv357776541:86 , srv37085901:86 , srv4452896:86 , srv496832077:86 , srv499324579:86 , srv534413083:86 , srv588476118:86 , srv690078825:86 , srv926962561:86 , srv1052609107:87 , srv113156719:87 , srv125528966:87 , srv1295223106:87 , srv1457515110:87 , srv1492911567:87 , srv1598000879:87 , srv1649424345:87 , srv1844084434:87 , srv1914794846:87 , srv1925625083:87 , srv209142571:87 , srv2114362880:87 , srv353690749:87 , srv520737830:87 , srv56022996:87 , srv982488167:87 , srv1454258926:88 , srv1480093961:88 , srv1757631410:88 , srv1939831338:88 , srv65746255:88 , srv731852234:88 , srv1151785161:89 , srv1668332594:89 , srv2125691280:89 , srv2058907738:530 }
2023-02-17 14:54:03,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-17 14:54:03,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1899385864=51, srv1539783001=36, srv534413083=84, srv807842089=95, srv1271006086=18, srv1305015618=24, srv1316181984=25, srv1852635107=48, srv1219541171=15, srv496832077=81, srv1911205155=52, srv627834816=90, srv1453080029=29, srv520737830=83, srv975320895=98, srv1247857880=16, srv1042970641=1, srv1372701682=26, srv1304085989=23, srv1179022036=13, srv125528966=17, srv2114362880=65, srv1668332594=42, srv162279861=40, srv2058907738=59, srv1858339455=49, srv1076178973=5, srv2125691280=66, srv1395793787=27, srv1202193951=14, srv1036204398=0, srv1157791936=10, srv1295223106=21, srv1455710708=31, srv1283819826=20, srv588476118=89, srv1770602525=45, srv470944088=79, srv953012760=97, srv37085901=75, srv319213505=71, srv1105262010=6, srv1757631410=44, srv982488167=99, srv1457515110=33, srv31048356=70, srv1912592226=53, srv2011531881=58, srv4452896=78, srv1939831338=56, srv690078825=92, srv1914794846=54, srv209142571=62, srv1454258926=30, srv127637808=19, srv499324579=82, srv1297753351=22, srv2064962851=60, srv56022996=86, srv1158272895=11, srv1407371508=28, srv1480093961=34, srv1598000879=39, srv2145183053=68, srv1176540314=12, srv1810191151=46, srv1925625083=55, srv1649424345=41, srv1591503979=37, srv357776541=74, srv328496183=72, srv731852234=94, srv583746224=88, srv1714728566=43, srv353690749=73, srv113156719=8, srv1992955522=57, srv477467692=80, srv392246547=77, srv1127988790=7, srv53779164=85, srv2111623629=64, srv1455984182=32, srv2089518820=61, srv1844084434=47, srv712386493=93, srv1052609107=2, srv1594322194=38, srv1492911567=35, srv1874773592=50, srv1069096274=3, srv65746255=91, srv926962561=96, srv1074889415=4, srv2130394908=67, srv562786161=87, srv379381707=76, srv1151785161=9, srv2103112576=63, srv244067715=69} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-17 14:54:03,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-17 14:54:03,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-17 14:54:03,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-17 14:54:03,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-17 14:54:03,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-17 14:54:03,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-17 14:54:03,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-17 14:54:03,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-17 14:54:03,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-17 14:54:03,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-17 14:54:03,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-17 14:54:03,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-17 14:54:03,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-17 14:54:03,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-17 14:54:03,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-17 14:54:03,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-17 14:54:03,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-17 14:54:03,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-17 14:54:03,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-17 14:54:03,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-17 14:54:03,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-17 14:54:03,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-17 14:54:03,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-17 14:54:03,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-17 14:54:03,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-17 14:54:03,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-17 14:54:03,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-17 14:54:03,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-17 14:54:03,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-17 14:54:03,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-17 14:54:03,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-17 14:54:03,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-17 14:54:03,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-17 14:54:03,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-17 14:54:03,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-17 14:54:03,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-17 14:54:03,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-17 14:54:03,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-17 14:54:03,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-17 14:54:03,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-17 14:54:03,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-17 14:54:03,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-17 14:54:03,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-17 14:54:03,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-17 14:54:03,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-17 14:54:03,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-17 14:54:03,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-17 14:54:03,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-17 14:54:03,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-17 14:54:03,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-17 14:54:03,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-17 14:54:03,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-17 14:54:03,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-17 14:54:03,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-17 14:54:03,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-17 14:54:03,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-17 14:54:03,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-17 14:54:03,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-17 14:54:03,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-17 14:54:03,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-17 14:54:03,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-17 14:54:03,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-17 14:54:03,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-17 14:54:03,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-17 14:54:03,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-17 14:54:03,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-17 14:54:03,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-17 14:54:03,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-17 14:54:03,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-17 14:54:03,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-17 14:54:03,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-17 14:54:03,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-17 14:54:03,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-17 14:54:03,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-17 14:54:03,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-17 14:54:03,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-17 14:54:03,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-17 14:54:03,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-17 14:54:03,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-17 14:54:03,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-17 14:54:03,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-17 14:54:03,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-17 14:54:03,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-17 14:54:03,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-17 14:54:03,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-17 14:54:03,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-17 14:54:03,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-17 14:54:03,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-17 14:54:03,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-17 14:54:03,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-17 14:54:03,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-17 14:54:03,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-17 14:54:03,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-17 14:54:03,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-17 14:54:03,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-17 14:54:03,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-17 14:54:03,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-17 14:54:03,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-17 14:54:03,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-17 14:54:03,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-17 14:54:03,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-17 14:54:03,983 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005885806956706312); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00704323668071124); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005885806956706312); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-17 14:54:03,983 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-17 14:54:03,983 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=5.927166895834437E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005885806956706312); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00704323668071124); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005885806956706312); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-17 14:54:04,329 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 458 ms to try 5000 different iterations.  Found a solution that moves 253 regions; Going from a computed imbalance of 5.927166895834437E-4 to a new imbalance of 1.8954400970760652E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.5313429231318235E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017727200782610348); MoveCostFunction : (multiplier=7.0, imbalance=0.02811111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.5313429231318235E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-17 14:54:04,330 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1036204398:83 , srv1480093961:83 , srv1649424345:83 , srv1668332594:83 , srv1810191151:83 , srv1852635107:83 , srv2114362880:83 , srv1042970641:84 , srv1052609107:84 , srv113156719:84 , srv1151785161:84 , srv125528966:84 , srv127637808:84 , srv1295223106:84 , srv1297753351:84 , srv1372701682:84 , srv1407371508:84 , srv1492911567:84 , srv1770602525:84 , srv1844084434:84 , srv1858339455:84 , srv1899385864:84 , srv1912592226:84 , srv1914794846:84 , srv1925625083:84 , srv1992955522:84 , srv2011531881:84 , srv2064962851:84 , srv209142571:84 , srv2111623629:84 , srv2125691280:84 , srv2145183053:84 , srv244067715:84 , srv328496183:84 , srv4452896:84 , srv496832077:84 , srv534413083:84 , srv56022996:84 , srv562786161:84 , srv583746224:84 , srv588476118:84 , srv627834816:84 , srv65746255:84 , srv712386493:84 , srv1069096274:85 , srv1074889415:85 , srv1076178973:85 , srv1105262010:85 , srv1127988790:85 , srv1157791936:85 , srv1158272895:85 , srv1176540314:85 , srv1179022036:85 , srv1202193951:85 , srv1219541171:85 , srv1247857880:85 , srv1283819826:85 , srv1304085989:85 , srv1316181984:85 , srv1395793787:85 , srv1453080029:85 , srv1454258926:85 , srv1455710708:85 , srv1455984182:85 , srv1457515110:85 , srv1539783001:85 , srv1591503979:85 , srv1594322194:85 , srv1598000879:85 , srv162279861:85 , srv1714728566:85 , srv1757631410:85 , srv1874773592:85 , srv1911205155:85 , srv2089518820:85 , srv2103112576:85 , srv2130394908:85 , srv31048356:85 , srv319213505:85 , srv353690749:85 , srv357776541:85 , srv37085901:85 , srv392246547:85 , srv470944088:85 , srv477467692:85 , srv499324579:85 , srv520737830:85 , srv53779164:85 , srv690078825:85 , srv731852234:85 , srv807842089:85 , srv926962561:85 , srv953012760:85 , srv975320895:85 , srv982488167:85 , srv1271006086:86 , srv1305015618:86 , srv1939831338:86 , srv379381707:86 , srv2058907738:632 }
2023-02-17 14:54:04,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-17 14:54:04,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1899385864=51, srv1539783001=36, srv534413083=84, srv807842089=95, srv1271006086=18, srv1305015618=24, srv1316181984=25, srv1852635107=48, srv1219541171=15, srv496832077=81, srv1911205155=52, srv627834816=90, srv1453080029=29, srv520737830=83, srv975320895=98, srv1247857880=16, srv1042970641=1, srv1372701682=26, srv1304085989=23, srv1179022036=13, srv125528966=17, srv2114362880=65, srv1668332594=42, srv162279861=40, srv2058907738=59, srv1858339455=49, srv1076178973=5, srv2125691280=66, srv1395793787=27, srv1202193951=14, srv1036204398=0, srv1157791936=10, srv1295223106=21, srv1455710708=31, srv1283819826=20, srv588476118=89, srv1770602525=45, srv470944088=79, srv953012760=97, srv37085901=75, srv319213505=71, srv1105262010=6, srv1757631410=44, srv982488167=99, srv1457515110=33, srv31048356=70, srv1912592226=53, srv2011531881=58, srv4452896=78, srv1939831338=56, srv690078825=92, srv1914794846=54, srv209142571=62, srv1454258926=30, srv127637808=19, srv499324579=82, srv1297753351=22, srv2064962851=60, srv56022996=86, srv1158272895=11, srv1407371508=28, srv1480093961=34, srv1598000879=39, srv2145183053=68, srv1176540314=12, srv1810191151=46, srv1925625083=55, srv1649424345=41, srv1591503979=37, srv357776541=74, srv328496183=72, srv731852234=94, srv583746224=88, srv1714728566=43, srv353690749=73, srv113156719=8, srv1992955522=57, srv477467692=80, srv392246547=77, srv1127988790=7, srv53779164=85, srv2111623629=64, srv1455984182=32, srv2089518820=61, srv1844084434=47, srv712386493=93, srv1052609107=2, srv1594322194=38, srv1492911567=35, srv1874773592=50, srv1069096274=3, srv65746255=91, srv926962561=96, srv1074889415=4, srv2130394908=67, srv562786161=87, srv379381707=76, srv1151785161=9, srv2103112576=63, srv244067715=69} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-17 14:54:04,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-17 14:54:04,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-17 14:54:04,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-17 14:54:04,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-17 14:54:04,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-17 14:54:04,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-17 14:54:04,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-17 14:54:04,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-17 14:54:04,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-17 14:54:04,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-17 14:54:04,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-17 14:54:04,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-17 14:54:04,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-17 14:54:04,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-17 14:54:04,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-17 14:54:04,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-17 14:54:04,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-17 14:54:04,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-17 14:54:04,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-17 14:54:04,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-17 14:54:04,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-17 14:54:04,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-17 14:54:04,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-17 14:54:04,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-17 14:54:04,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-17 14:54:04,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-17 14:54:04,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-17 14:54:04,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-17 14:54:04,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-17 14:54:04,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-17 14:54:04,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-17 14:54:04,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-17 14:54:04,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-17 14:54:04,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-17 14:54:04,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-17 14:54:04,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-17 14:54:04,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-17 14:54:04,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-17 14:54:04,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-17 14:54:04,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-17 14:54:04,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-17 14:54:04,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-17 14:54:04,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-17 14:54:04,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-17 14:54:04,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-17 14:54:04,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-17 14:54:04,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-17 14:54:04,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-17 14:54:04,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-17 14:54:04,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-17 14:54:04,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-17 14:54:04,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-17 14:54:04,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-17 14:54:04,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-17 14:54:04,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-17 14:54:04,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-17 14:54:04,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-17 14:54:04,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-17 14:54:04,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-17 14:54:04,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-17 14:54:04,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-17 14:54:04,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-17 14:54:04,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-17 14:54:04,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-17 14:54:04,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-17 14:54:04,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-17 14:54:04,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-17 14:54:04,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-17 14:54:04,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-17 14:54:04,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-17 14:54:04,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-17 14:54:04,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-17 14:54:04,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-17 14:54:04,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-17 14:54:04,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-17 14:54:04,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-17 14:54:04,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-17 14:54:04,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-17 14:54:04,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-17 14:54:04,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-17 14:54:04,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-17 14:54:04,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-17 14:54:04,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-17 14:54:04,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-17 14:54:04,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-17 14:54:04,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-17 14:54:04,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-17 14:54:04,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-17 14:54:04,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-17 14:54:04,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-17 14:54:04,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-17 14:54:04,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-17 14:54:04,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-17 14:54:04,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-17 14:54:04,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-17 14:54:04,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-17 14:54:04,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-17 14:54:04,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-17 14:54:04,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-17 14:54:04,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-17 14:54:04,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-17 14:54:04,462 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.7182857106794416E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.5313429231318235E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017727200782610348); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.5313429231318235E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-17 14:54:04,463 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1036204398:83 , srv1480093961:83 , srv1649424345:83 , srv1668332594:83 , srv1810191151:83 , srv1852635107:83 , srv2114362880:83 , srv1042970641:84 , srv1052609107:84 , srv113156719:84 , srv1151785161:84 , srv125528966:84 , srv127637808:84 , srv1295223106:84 , srv1297753351:84 , srv1372701682:84 , srv1407371508:84 , srv1492911567:84 , srv1770602525:84 , srv1844084434:84 , srv1858339455:84 , srv1899385864:84 , srv1912592226:84 , srv1914794846:84 , srv1925625083:84 , srv1992955522:84 , srv2011531881:84 , srv2064962851:84 , srv209142571:84 , srv2111623629:84 , srv2125691280:84 , srv2145183053:84 , srv244067715:84 , srv328496183:84 , srv4452896:84 , srv496832077:84 , srv534413083:84 , srv56022996:84 , srv562786161:84 , srv583746224:84 , srv588476118:84 , srv627834816:84 , srv65746255:84 , srv712386493:84 , srv1069096274:85 , srv1074889415:85 , srv1076178973:85 , srv1105262010:85 , srv1127988790:85 , srv1157791936:85 , srv1158272895:85 , srv1176540314:85 , srv1179022036:85 , srv1202193951:85 , srv1219541171:85 , srv1247857880:85 , srv1283819826:85 , srv1304085989:85 , srv1316181984:85 , srv1395793787:85 , srv1453080029:85 , srv1454258926:85 , srv1455710708:85 , srv1455984182:85 , srv1457515110:85 , srv1539783001:85 , srv1591503979:85 , srv1594322194:85 , srv1598000879:85 , srv162279861:85 , srv1714728566:85 , srv1757631410:85 , srv1874773592:85 , srv1911205155:85 , srv2089518820:85 , srv2103112576:85 , srv2130394908:85 , srv31048356:85 , srv319213505:85 , srv353690749:85 , srv357776541:85 , srv37085901:85 , srv392246547:85 , srv470944088:85 , srv477467692:85 , srv499324579:85 , srv520737830:85 , srv53779164:85 , srv690078825:85 , srv731852234:85 , srv807842089:85 , srv926962561:85 , srv953012760:85 , srv975320895:85 , srv982488167:85 , srv1271006086:86 , srv1305015618:86 , srv1939831338:86 , srv379381707:86 , srv2058907738:632 }
2023-02-17 14:54:04,493 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=297 (was 270) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=8359 (was 8849)
2023-02-17 14:54:04,502 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=297, ProcessCount=170, AvailableMemoryMB=8359
2023-02-17 14:54:04,503 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-17 14:54:04,503 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-02-17 14:54:04,511 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1018833890:84 , srv1050721486:84 , srv1054505956:84 , srv1062738918:84 , srv1295186754:84 , srv1312523372:84 , srv1384112286:84 , srv1466570129:84 , srv1552284885:84 , srv1567799667:84 , srv170073712:84 , srv1772539956:84 , srv1790856832:84 , srv1807953878:84 , srv1863052522:84 , srv2135138826:84 , srv2146801589:84 , srv248407396:84 , srv294300434:84 , srv326284772:84 , srv329733001:84 , srv394469990:84 , srv416119388:84 , srv459732216:84 , srv598712032:84 , srv65781515:84 , srv678628900:84 , srv71183669:84 , srv727251435:84 , srv759012904:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-02-17 14:54:04,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-17 14:54:04,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1054505956=2, srv1772539956=11, srv459732216=23, srv1018833890=0, srv416119388=22, srv1552284885=8, srv678628900=26, srv1863052522=14, srv170073712=10, srv1790856832=12, srv727251435=28, srv759012904=29, srv1807953878=13, srv1295186754=4, srv2135138826=15, srv248407396=17, srv71183669=27, srv394469990=21, srv326284772=19, srv1062738918=3, srv1050721486=1, srv1466570129=7, srv1384112286=6, srv65781515=25, srv1567799667=9, srv598712032=24, srv294300434=18, srv329733001=20, srv2146801589=16, srv1312523372=5} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-17 14:54:04,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-17 14:54:04,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-17 14:54:04,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-17 14:54:04,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-17 14:54:04,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-17 14:54:04,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-17 14:54:04,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-17 14:54:04,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-17 14:54:04,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-17 14:54:04,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-17 14:54:04,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-17 14:54:04,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-17 14:54:04,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-17 14:54:04,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-17 14:54:04,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-17 14:54:04,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-17 14:54:04,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-17 14:54:04,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-17 14:54:04,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-17 14:54:04,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-17 14:54:04,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-17 14:54:04,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-17 14:54:04,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-17 14:54:04,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-17 14:54:04,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-17 14:54:04,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-17 14:54:04,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-17 14:54:04,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-17 14:54:04,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-17 14:54:04,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-17 14:54:04,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-02-17 14:54:04,537 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1485411769444585, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-17 14:54:04,537 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912661091662516, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1485411769444585, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-02-17 14:56:04,525 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6123290 different iterations.  Found a solution that moves 1854 regions; Going from a computed imbalance of 0.9912661091662516 to a new imbalance of 4.3273284898463826E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6866666666666666, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-17 14:56:04,532 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1018833890:90 , srv1050721486:90 , srv1054505956:90 , srv1062738918:90 , srv1295186754:90 , srv1312523372:90 , srv1384112286:90 , srv1466570129:90 , srv1552284885:90 , srv1567799667:90 , srv170073712:90 , srv1772539956:90 , srv1790856832:90 , srv1807953878:90 , srv1863052522:90 , srv2135138826:90 , srv2146801589:90 , srv248407396:90 , srv294300434:90 , srv326284772:90 , srv329733001:90 , srv394469990:90 , srv416119388:90 , srv459732216:90 , srv598712032:90 , srv65781515:90 , srv678628900:90 , srv71183669:90 , srv727251435:90 , srv759012904:90 }
2023-02-17 14:56:04,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-17 14:56:04,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1054505956=2, srv1772539956=11, srv459732216=23, srv1018833890=0, srv416119388=22, srv1552284885=8, srv678628900=26, srv1863052522=14, srv170073712=10, srv1790856832=12, srv727251435=28, srv759012904=29, srv1807953878=13, srv1295186754=4, srv2135138826=15, srv248407396=17, srv71183669=27, srv394469990=21, srv326284772=19, srv1062738918=3, srv1050721486=1, srv1466570129=7, srv1384112286=6, srv65781515=25, srv1567799667=9, srv598712032=24, srv294300434=18, srv329733001=20, srv2146801589=16, srv1312523372=5} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-17 14:56:04,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-17 14:56:04,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-17 14:56:04,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-17 14:56:04,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-17 14:56:04,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-17 14:56:04,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-17 14:56:04,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-17 14:56:04,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-17 14:56:04,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-17 14:56:04,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-17 14:56:04,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-17 14:56:04,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-17 14:56:04,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-17 14:56:04,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-17 14:56:04,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-17 14:56:04,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-17 14:56:04,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-17 14:56:04,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-17 14:56:04,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-17 14:56:04,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-17 14:56:04,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-17 14:56:04,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-17 14:56:04,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-17 14:56:04,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-17 14:56:04,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-17 14:56:04,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-17 14:56:04,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-17 14:56:04,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-17 14:56:04,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-17 14:56:04,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-17 14:56:04,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-02-17 14:56:04,552 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-17 14:56:04,552 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1018833890:90 , srv1050721486:90 , srv1054505956:90 , srv1062738918:90 , srv1295186754:90 , srv1312523372:90 , srv1384112286:90 , srv1466570129:90 , srv1552284885:90 , srv1567799667:90 , srv170073712:90 , srv1772539956:90 , srv1790856832:90 , srv1807953878:90 , srv1863052522:90 , srv2135138826:90 , srv2146801589:90 , srv248407396:90 , srv294300434:90 , srv326284772:90 , srv329733001:90 , srv394469990:90 , srv416119388:90 , srv459732216:90 , srv598712032:90 , srv65781515:90 , srv678628900:90 , srv71183669:90 , srv727251435:90 , srv759012904:90 }
2023-02-17 14:56:04,573 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=176 (was 297), ProcessCount=170 (was 170), AvailableMemoryMB=8190 (was 8359)