2023-04-24 13:53:34,384 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-04-24 13:53:34,659 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-04-24 13:53:34,754 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-04-24 13:53:34,810 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-24 13:53:34,817 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-04-24 13:53:34,830 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=404, ProcessCount=170, AvailableMemoryMB=8998
2023-04-24 13:53:34,832 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-24 13:53:34,832 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-04-24 13:53:34,948 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1007938144:84 , srv1049888292:84 , srv1085207044:84 , srv111174730:84 , srv1112756844:84 , srv1124506456:84 , srv1135719273:84 , srv1177536125:84 , srv122073948:84 , srv1275946718:84 , srv1295387511:84 , srv1296423549:84 , srv1349217485:84 , srv1382981855:84 , srv1390020735:84 , srv1395364996:84 , srv1396174741:84 , srv1435682949:84 , srv1440059620:84 , srv1451689622:84 , srv1484810033:84 , srv151452705:84 , srv1560863829:84 , srv1563407380:84 , srv1563577225:84 , srv1577149768:84 , srv1618243627:84 , srv16256222:84 , srv1642580980:84 , srv1654143018:84 , srv166108406:84 , srv1686719182:84 , srv1696198163:84 , srv1718635216:84 , srv172732176:84 , srv1760441402:84 , srv176484889:84 , srv1805310755:84 , srv1810158322:84 , srv1825463833:84 , srv1827453345:84 , srv1850575555:84 , srv1850933261:84 , srv1851587644:84 , srv1874348089:84 , srv1881815539:84 , srv1917560528:84 , srv191837390:84 , srv1970667114:84 , srv1987567038:84 , srv2010836640:84 , srv2012561161:84 , srv2067681382:84 , srv2082522020:84 , srv2090454543:84 , srv210298266:84 , srv2118787632:84 , srv213477239:84 , srv2137040705:84 , srv2137861069:84 , srv2139824659:84 , srv219967186:84 , srv243150057:84 , srv286122172:84 , srv338810102:84 , srv356094770:84 , srv362015264:84 , srv370784493:84 , srv384923833:84 , srv393923538:84 , srv43702921:84 , srv440275323:84 , srv44095414:84 , srv451274144:84 , srv52057865:84 , srv536469789:84 , srv542874362:84 , srv562153112:84 , srv576942190:84 , srv581594779:84 , srv59926084:84 , srv652772091:84 , srv655815372:84 , srv659166197:84 , srv671329866:84 , srv721886421:84 , srv76372558:84 , srv812999447:84 , srv832034497:84 , srv871001218:84 , srv882082258:84 , srv891802918:84 , srv914233195:84 , srv916587316:84 , srv961248162:84 , srv963171803:84 , srv963450734:84 , srv970174018:84 , srv997980367:84 , srv528017155:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-04-24 13:53:34,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-24 13:53:34,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv111174730=3, srv1881815539=45, srv970174018=98, srv1760441402=35, srv2067681382=52, srv1851587644=43, srv1295387511=10, srv581594779=80, srv871001218=90, srv1642580980=28, srv914233195=93, srv1560863829=22, srv2082522020=53, srv542874362=77, srv963450734=97, srv1112756844=4, srv1435682949=17, srv2137040705=58, srv891802918=92, srv76372558=87, srv671329866=85, srv243150057=62, srv362015264=66, srv1577149768=25, srv1696198163=32, srv562153112=78, srv576942190=79, srv2137861069=59, srv2010836640=50, srv1395364996=15, srv882082258=91, srv1349217485=12, srv210298266=55, srv338810102=64, srv1007938144=0, srv356094770=65, srv1850575555=41, srv1440059620=18, srv1827453345=40, srv172732176=34, srv721886421=86, srv1124506456=5, srv2118787632=56, srv1850933261=42, srv384923833=68, srv1654143018=29, srv43702921=70, srv832034497=89, srv963171803=96, srv440275323=71, srv1135719273=6, srv166108406=30, srv536469789=76, srv370784493=67, srv1451689622=19, srv812999447=88, srv1296423549=11, srv1810158322=38, srv1917560528=46, srv2090454543=54, srv59926084=81, srv1484810033=20, srv1686719182=31, srv1396174741=16, srv655815372=83, srv44095414=72, srv2139824659=60, srv997980367=99, srv219967186=61, srv393923538=69, srv1177536125=7, srv1049888292=1, srv1085207044=2, srv1718635216=33, srv916587316=94, srv286122172=63, srv176484889=36, srv1825463833=39, srv451274144=73, srv961248162=95, srv1970667114=48, srv1382981855=13, srv52057865=74, srv151452705=21, srv652772091=82, srv1563577225=24, srv1987567038=49, srv122073948=8, srv1563407380=23, srv1874348089=44, srv191837390=47, srv16256222=27, srv528017155=75, srv1275946718=9, srv1618243627=26, srv213477239=57, srv2012561161=51, srv1805310755=37, srv1390020735=14, srv659166197=84} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-24 13:53:35,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-24 13:53:35,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-24 13:53:35,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-24 13:53:35,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-24 13:53:35,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-24 13:53:35,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-24 13:53:35,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-24 13:53:35,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-24 13:53:35,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-24 13:53:35,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-24 13:53:35,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-24 13:53:35,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-24 13:53:35,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-24 13:53:35,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-24 13:53:35,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-24 13:53:35,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-24 13:53:35,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-24 13:53:35,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-24 13:53:35,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-24 13:53:35,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-24 13:53:35,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-24 13:53:35,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-24 13:53:35,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-24 13:53:35,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-24 13:53:35,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-24 13:53:35,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-24 13:53:35,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-24 13:53:35,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-24 13:53:35,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-24 13:53:35,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-24 13:53:35,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-24 13:53:35,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-24 13:53:35,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-24 13:53:35,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-24 13:53:35,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-24 13:53:35,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-24 13:53:35,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-24 13:53:35,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-24 13:53:35,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-24 13:53:35,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-24 13:53:35,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-24 13:53:35,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-24 13:53:35,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-24 13:53:35,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-24 13:53:35,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-24 13:53:35,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-24 13:53:35,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-24 13:53:35,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-24 13:53:35,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-24 13:53:35,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-24 13:53:35,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-24 13:53:35,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-24 13:53:35,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-24 13:53:35,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-24 13:53:35,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-24 13:53:35,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-24 13:53:35,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-24 13:53:35,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-24 13:53:35,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-24 13:53:35,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-24 13:53:35,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-24 13:53:35,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-24 13:53:35,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-24 13:53:35,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-24 13:53:35,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-24 13:53:35,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-24 13:53:35,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-24 13:53:35,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-24 13:53:35,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-24 13:53:35,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-24 13:53:35,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-24 13:53:35,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-24 13:53:35,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-24 13:53:35,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-24 13:53:35,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-24 13:53:35,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-24 13:53:35,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-24 13:53:35,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-24 13:53:35,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-24 13:53:35,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-24 13:53:35,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-24 13:53:35,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-24 13:53:35,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-24 13:53:35,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-24 13:53:35,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-24 13:53:35,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-24 13:53:35,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-24 13:53:35,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-24 13:53:35,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-24 13:53:35,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-24 13:53:35,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-24 13:53:35,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-24 13:53:35,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-24 13:53:35,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-24 13:53:35,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-24 13:53:35,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-24 13:53:35,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-24 13:53:35,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-24 13:53:35,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-24 13:53:35,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-24 13:53:35,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-24 13:53:35,181 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-24 13:53:35,182 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-24 13:53:35,182 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-24 13:53:35,427 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 346 ms to try 5000 different iterations.  Found a solution that moves 4669 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18385116894996784. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03962317830367912, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0569462305706577, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5187777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03962317830367912, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17966666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24016666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-24 13:53:35,456 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1917560528:81 , srv2118787632:81 , srv1850575555:82 , srv384923833:82 , srv659166197:82 , srv1390020735:83 , srv1484810033:83 , srv1827453345:83 , srv1987567038:83 , srv2067681382:83 , srv451274144:83 , srv542874362:83 , srv891802918:83 , srv916587316:83 , srv1112756844:84 , srv1275946718:84 , srv1577149768:84 , srv1810158322:84 , srv1850933261:84 , srv1851587644:84 , srv1874348089:84 , srv191837390:84 , srv2137040705:84 , srv44095414:84 , srv832034497:84 , srv1085207044:85 , srv1124506456:85 , srv122073948:85 , srv1382981855:85 , srv16256222:85 , srv1654143018:85 , srv1881815539:85 , srv2012561161:85 , srv213477239:85 , srv362015264:85 , srv576942190:85 , srv721886421:85 , srv882082258:85 , srv1049888292:86 , srv1349217485:86 , srv1396174741:86 , srv1563407380:86 , srv1618243627:86 , srv1825463833:86 , srv219967186:86 , srv243150057:86 , srv338810102:86 , srv393923538:86 , srv652772091:86 , srv914233195:86 , srv963450734:86 , srv1295387511:87 , srv1435682949:87 , srv1718635216:87 , srv172732176:87 , srv176484889:87 , srv1805310755:87 , srv2010836640:87 , srv2090454543:87 , srv2137861069:87 , srv440275323:87 , srv536469789:87 , srv581594779:87 , srv59926084:87 , srv655815372:87 , srv671329866:87 , srv76372558:87 , srv812999447:87 , srv961248162:87 , srv963171803:87 , srv1177536125:88 , srv1563577225:88 , srv1686719182:88 , srv1760441402:88 , srv370784493:88 , srv52057865:88 , srv562153112:88 , srv871001218:88 , srv997980367:88 , srv1007938144:89 , srv1296423549:89 , srv1642580980:89 , srv1970667114:89 , srv356094770:89 , srv43702921:89 , srv970174018:89 , srv111174730:90 , srv1395364996:90 , srv151452705:90 , srv1696198163:90 , srv2082522020:90 , srv1135719273:91 , srv1440059620:91 , srv2139824659:91 , srv1560863829:92 , srv166108406:92 , srv286122172:92 , srv210298266:94 , srv1451689622:95 , srv528017155:442 }
2023-04-24 13:53:35,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-24 13:53:35,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv111174730=3, srv1881815539=45, srv970174018=98, srv1760441402=35, srv2067681382=52, srv1851587644=43, srv1295387511=10, srv581594779=80, srv871001218=90, srv1642580980=28, srv914233195=93, srv1560863829=22, srv2082522020=53, srv542874362=77, srv963450734=97, srv1112756844=4, srv1435682949=17, srv2137040705=58, srv891802918=92, srv76372558=87, srv671329866=85, srv243150057=62, srv362015264=66, srv1577149768=25, srv1696198163=32, srv562153112=78, srv576942190=79, srv2137861069=59, srv2010836640=50, srv1395364996=15, srv882082258=91, srv1349217485=12, srv210298266=55, srv338810102=64, srv1007938144=0, srv356094770=65, srv1850575555=41, srv1440059620=18, srv1827453345=40, srv172732176=34, srv721886421=86, srv1124506456=5, srv2118787632=56, srv1850933261=42, srv384923833=68, srv1654143018=29, srv43702921=70, srv832034497=89, srv963171803=96, srv440275323=71, srv1135719273=6, srv166108406=30, srv536469789=76, srv370784493=67, srv1451689622=19, srv812999447=88, srv1296423549=11, srv1810158322=38, srv1917560528=46, srv2090454543=54, srv59926084=81, srv1484810033=20, srv1686719182=31, srv1396174741=16, srv655815372=83, srv44095414=72, srv2139824659=60, srv997980367=99, srv219967186=61, srv393923538=69, srv1177536125=7, srv1049888292=1, srv1085207044=2, srv1718635216=33, srv916587316=94, srv286122172=63, srv176484889=36, srv1825463833=39, srv451274144=73, srv961248162=95, srv1970667114=48, srv1382981855=13, srv52057865=74, srv151452705=21, srv652772091=82, srv1563577225=24, srv1987567038=49, srv122073948=8, srv1563407380=23, srv1874348089=44, srv191837390=47, srv16256222=27, srv528017155=75, srv1275946718=9, srv1618243627=26, srv213477239=57, srv2012561161=51, srv1805310755=37, srv1390020735=14, srv659166197=84} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-24 13:53:35,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-24 13:53:35,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-24 13:53:35,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-24 13:53:35,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-24 13:53:35,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-24 13:53:35,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-24 13:53:35,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-24 13:53:35,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-24 13:53:35,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-24 13:53:35,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-24 13:53:35,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-24 13:53:35,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-24 13:53:35,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-24 13:53:35,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-24 13:53:35,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-24 13:53:35,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-24 13:53:35,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-24 13:53:35,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-24 13:53:35,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-24 13:53:35,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-24 13:53:35,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-24 13:53:35,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-24 13:53:35,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-24 13:53:35,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-24 13:53:35,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-24 13:53:35,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-24 13:53:35,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-24 13:53:35,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-24 13:53:35,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-24 13:53:35,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-24 13:53:35,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-24 13:53:35,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-24 13:53:35,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-24 13:53:35,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-24 13:53:35,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-24 13:53:35,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-24 13:53:35,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-24 13:53:35,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-24 13:53:35,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-24 13:53:35,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-24 13:53:35,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-24 13:53:35,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-24 13:53:35,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-24 13:53:35,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-24 13:53:35,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-24 13:53:35,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-24 13:53:35,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-24 13:53:35,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-24 13:53:35,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-24 13:53:35,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-24 13:53:35,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-24 13:53:35,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-24 13:53:35,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-24 13:53:35,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-24 13:53:35,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-24 13:53:35,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-24 13:53:35,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-24 13:53:35,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-24 13:53:35,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-24 13:53:35,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-24 13:53:35,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-24 13:53:35,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-24 13:53:35,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-24 13:53:35,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-24 13:53:35,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-24 13:53:35,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-24 13:53:35,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-24 13:53:35,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-24 13:53:35,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-24 13:53:35,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-24 13:53:35,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-24 13:53:35,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-24 13:53:35,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-24 13:53:35,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-24 13:53:35,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-24 13:53:35,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-24 13:53:35,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-24 13:53:35,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-24 13:53:35,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-24 13:53:35,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-24 13:53:35,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-24 13:53:35,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-24 13:53:35,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-24 13:53:35,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-24 13:53:35,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-24 13:53:35,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-24 13:53:35,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-24 13:53:35,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-24 13:53:35,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-24 13:53:35,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-24 13:53:35,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-24 13:53:35,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-24 13:53:35,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-24 13:53:35,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-24 13:53:35,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-24 13:53:35,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-24 13:53:35,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-24 13:53:35,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-24 13:53:35,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-24 13:53:35,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-24 13:53:35,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-24 13:53:35,583 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03962317830367912, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0569462305706577, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03962317830367912, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17966666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24016666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-24 13:53:35,583 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-24 13:53:35,583 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18381847591320555, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03962317830367912, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0569462305706577, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03962317830367912, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17966666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24016666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-24 13:53:35,794 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 298 ms to try 5000 different iterations.  Found a solution that moves 2660 regions; Going from a computed imbalance of 0.18381847591320555 to a new imbalance of 0.014689298923305033. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022842630295629177); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03970712081380504, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29555555555555557, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022842630295629177); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01175); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04225, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-24 13:53:35,805 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1451689622:75 , srv111174730:79 , srv151452705:79 , srv210298266:79 , srv2082522020:80 , srv286122172:80 , srv1135719273:81 , srv1560863829:81 , srv1696198163:81 , srv1440059620:82 , srv1642580980:82 , srv166108406:82 , srv1970667114:82 , srv2139824659:82 , srv440275323:82 , srv1007938144:83 , srv1295387511:83 , srv1349217485:83 , srv1395364996:83 , srv1686719182:83 , srv1760441402:83 , srv2137861069:83 , srv243150057:83 , srv356094770:83 , srv970174018:83 , srv1177536125:84 , srv122073948:84 , srv1296423549:84 , srv1435682949:84 , srv1718635216:84 , srv370784493:84 , srv52057865:84 , srv536469789:84 , srv562153112:84 , srv59926084:84 , srv871001218:84 , srv961248162:84 , srv1049888292:85 , srv1396174741:85 , srv172732176:85 , srv2010836640:85 , srv2012561161:85 , srv43702921:85 , srv581594779:85 , srv652772091:85 , srv655815372:85 , srv914233195:85 , srv963171803:85 , srv963450734:85 , srv1124506456:86 , srv16256222:86 , srv176484889:86 , srv1805310755:86 , srv1881815539:86 , srv2090454543:86 , srv576942190:86 , srv671329866:86 , srv721886421:86 , srv76372558:86 , srv997980367:86 , srv1085207044:87 , srv1112756844:87 , srv1563407380:87 , srv1563577225:87 , srv1618243627:87 , srv1825463833:87 , srv1827453345:87 , srv338810102:87 , srv451274144:87 , srv812999447:87 , srv1275946718:88 , srv1382981855:88 , srv1484810033:88 , srv1577149768:88 , srv1654143018:88 , srv1810158322:88 , srv1874348089:88 , srv191837390:88 , srv1987567038:88 , srv2067681382:88 , srv213477239:88 , srv393923538:88 , srv44095414:88 , srv891802918:88 , srv1850575555:89 , srv2137040705:89 , srv219967186:89 , srv659166197:89 , srv832034497:89 , srv882082258:89 , srv1390020735:90 , srv1850933261:90 , srv362015264:90 , srv384923833:90 , srv542874362:90 , srv916587316:90 , srv1917560528:92 , srv1851587644:93 , srv2118787632:93 , srv528017155:535 }
2023-04-24 13:53:35,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-24 13:53:35,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv111174730=3, srv1881815539=45, srv970174018=98, srv1760441402=35, srv2067681382=52, srv1851587644=43, srv1295387511=10, srv581594779=80, srv871001218=90, srv1642580980=28, srv914233195=93, srv1560863829=22, srv2082522020=53, srv542874362=77, srv963450734=97, srv1112756844=4, srv1435682949=17, srv2137040705=58, srv891802918=92, srv76372558=87, srv671329866=85, srv243150057=62, srv362015264=66, srv1577149768=25, srv1696198163=32, srv562153112=78, srv576942190=79, srv2137861069=59, srv2010836640=50, srv1395364996=15, srv882082258=91, srv1349217485=12, srv210298266=55, srv338810102=64, srv1007938144=0, srv356094770=65, srv1850575555=41, srv1440059620=18, srv1827453345=40, srv172732176=34, srv721886421=86, srv1124506456=5, srv2118787632=56, srv1850933261=42, srv384923833=68, srv1654143018=29, srv43702921=70, srv832034497=89, srv963171803=96, srv440275323=71, srv1135719273=6, srv166108406=30, srv536469789=76, srv370784493=67, srv1451689622=19, srv812999447=88, srv1296423549=11, srv1810158322=38, srv1917560528=46, srv2090454543=54, srv59926084=81, srv1484810033=20, srv1686719182=31, srv1396174741=16, srv655815372=83, srv44095414=72, srv2139824659=60, srv997980367=99, srv219967186=61, srv393923538=69, srv1177536125=7, srv1049888292=1, srv1085207044=2, srv1718635216=33, srv916587316=94, srv286122172=63, srv176484889=36, srv1825463833=39, srv451274144=73, srv961248162=95, srv1970667114=48, srv1382981855=13, srv52057865=74, srv151452705=21, srv652772091=82, srv1563577225=24, srv1987567038=49, srv122073948=8, srv1563407380=23, srv1874348089=44, srv191837390=47, srv16256222=27, srv528017155=75, srv1275946718=9, srv1618243627=26, srv213477239=57, srv2012561161=51, srv1805310755=37, srv1390020735=14, srv659166197=84} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-24 13:53:35,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-24 13:53:35,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-24 13:53:35,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-24 13:53:35,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-24 13:53:35,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-24 13:53:35,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-24 13:53:35,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-24 13:53:35,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-24 13:53:35,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-24 13:53:35,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-24 13:53:35,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-24 13:53:35,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-24 13:53:35,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-24 13:53:35,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-24 13:53:35,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-24 13:53:35,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-24 13:53:35,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-24 13:53:35,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-24 13:53:35,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-24 13:53:35,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-24 13:53:35,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-24 13:53:35,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-24 13:53:35,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-24 13:53:35,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-24 13:53:35,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-24 13:53:35,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-24 13:53:35,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-24 13:53:35,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-24 13:53:35,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-24 13:53:35,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-24 13:53:35,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-24 13:53:35,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-24 13:53:35,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-24 13:53:35,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-24 13:53:35,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-24 13:53:35,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-24 13:53:35,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-24 13:53:35,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-24 13:53:35,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-24 13:53:35,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-24 13:53:35,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-24 13:53:35,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-24 13:53:35,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-24 13:53:35,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-24 13:53:35,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-24 13:53:35,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-24 13:53:35,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-24 13:53:35,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-24 13:53:35,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-24 13:53:35,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-24 13:53:35,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-24 13:53:35,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-24 13:53:35,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-24 13:53:35,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-24 13:53:35,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-24 13:53:35,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-24 13:53:35,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-24 13:53:35,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-24 13:53:35,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-24 13:53:35,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-24 13:53:35,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-24 13:53:35,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-24 13:53:35,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-24 13:53:35,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-24 13:53:35,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-24 13:53:35,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-24 13:53:35,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-24 13:53:35,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-24 13:53:35,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-24 13:53:35,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-24 13:53:35,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-24 13:53:35,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-24 13:53:35,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-24 13:53:35,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-24 13:53:35,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-24 13:53:35,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-24 13:53:35,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-24 13:53:35,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-24 13:53:35,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-24 13:53:35,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-24 13:53:35,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-24 13:53:35,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-24 13:53:35,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-24 13:53:35,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-24 13:53:35,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-24 13:53:35,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-24 13:53:35,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-24 13:53:35,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-24 13:53:35,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-24 13:53:35,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-24 13:53:35,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-24 13:53:35,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-24 13:53:35,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-24 13:53:35,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-24 13:53:35,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-24 13:53:35,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-24 13:53:35,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-24 13:53:35,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-24 13:53:35,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-24 13:53:35,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-24 13:53:35,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-24 13:53:35,918 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022842630295629177); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03970712081380504, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022842630295629177); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01175); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04225, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-24 13:53:35,918 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-24 13:53:35,919 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.014670673205209577, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022842630295629177); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03970712081380504, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022842630295629177); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01175); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04225, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-24 13:53:36,085 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 239 ms to try 5000 different iterations.  Found a solution that moves 960 regions; Going from a computed imbalance of 0.014670673205209577 to a new imbalance of 6.722638089146552E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006104220173631566); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0113212452865072); MoveCostFunction : (multiplier=7.0, imbalance=0.10666666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006104220173631566); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-24 13:53:36,088 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1851587644:81 , srv219967186:82 , srv362015264:82 , srv1563577225:83 , srv1825463833:83 , srv1850933261:83 , srv338810102:83 , srv812999447:83 , srv1296423549:84 , srv1390020735:84 , srv1560863829:84 , srv1563407380:84 , srv1618243627:84 , srv1654143018:84 , srv166108406:84 , srv176484889:84 , srv1805310755:84 , srv2090454543:84 , srv210298266:84 , srv2118787632:84 , srv213477239:84 , srv2137040705:84 , srv393923538:84 , srv43702921:84 , srv671329866:84 , srv76372558:84 , srv882082258:84 , srv1085207044:85 , srv1135719273:85 , srv1275946718:85 , srv1382981855:85 , srv1395364996:85 , srv1577149768:85 , srv1760441402:85 , srv1810158322:85 , srv1917560528:85 , srv2010836640:85 , srv2139824659:85 , srv356094770:85 , srv370784493:85 , srv44095414:85 , srv52057865:85 , srv832034497:85 , srv871001218:85 , srv963171803:85 , srv997980367:85 , srv1007938144:86 , srv1112756844:86 , srv1124506456:86 , srv1177536125:86 , srv1440059620:86 , srv1484810033:86 , srv16256222:86 , srv1642580980:86 , srv172732176:86 , srv1850575555:86 , srv1874348089:86 , srv1881815539:86 , srv191837390:86 , srv2067681382:86 , srv286122172:86 , srv384923833:86 , srv542874362:86 , srv581594779:86 , srv652772091:86 , srv655815372:86 , srv659166197:86 , srv721886421:86 , srv891802918:86 , srv914233195:86 , srv916587316:86 , srv963450734:86 , srv970174018:86 , srv1049888292:87 , srv1295387511:87 , srv1349217485:87 , srv1396174741:87 , srv1435682949:87 , srv1686719182:87 , srv1696198163:87 , srv1718635216:87 , srv1970667114:87 , srv1987567038:87 , srv451274144:87 , srv536469789:87 , srv562153112:87 , srv576942190:87 , srv59926084:87 , srv961248162:87 , srv111174730:88 , srv122073948:88 , srv1451689622:88 , srv151452705:88 , srv1827453345:88 , srv2012561161:88 , srv2082522020:88 , srv2137861069:88 , srv243150057:88 , srv440275323:88 , srv528017155:535 }
2023-04-24 13:53:36,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-24 13:53:36,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv111174730=3, srv1881815539=45, srv970174018=98, srv1760441402=35, srv2067681382=52, srv1851587644=43, srv1295387511=10, srv581594779=80, srv871001218=90, srv1642580980=28, srv914233195=93, srv1560863829=22, srv2082522020=53, srv542874362=77, srv963450734=97, srv1112756844=4, srv1435682949=17, srv2137040705=58, srv891802918=92, srv76372558=87, srv671329866=85, srv243150057=62, srv362015264=66, srv1577149768=25, srv1696198163=32, srv562153112=78, srv576942190=79, srv2137861069=59, srv2010836640=50, srv1395364996=15, srv882082258=91, srv1349217485=12, srv210298266=55, srv338810102=64, srv1007938144=0, srv356094770=65, srv1850575555=41, srv1440059620=18, srv1827453345=40, srv172732176=34, srv721886421=86, srv1124506456=5, srv2118787632=56, srv1850933261=42, srv384923833=68, srv1654143018=29, srv43702921=70, srv832034497=89, srv963171803=96, srv440275323=71, srv1135719273=6, srv166108406=30, srv536469789=76, srv370784493=67, srv1451689622=19, srv812999447=88, srv1296423549=11, srv1810158322=38, srv1917560528=46, srv2090454543=54, srv59926084=81, srv1484810033=20, srv1686719182=31, srv1396174741=16, srv655815372=83, srv44095414=72, srv2139824659=60, srv997980367=99, srv219967186=61, srv393923538=69, srv1177536125=7, srv1049888292=1, srv1085207044=2, srv1718635216=33, srv916587316=94, srv286122172=63, srv176484889=36, srv1825463833=39, srv451274144=73, srv961248162=95, srv1970667114=48, srv1382981855=13, srv52057865=74, srv151452705=21, srv652772091=82, srv1563577225=24, srv1987567038=49, srv122073948=8, srv1563407380=23, srv1874348089=44, srv191837390=47, srv16256222=27, srv528017155=75, srv1275946718=9, srv1618243627=26, srv213477239=57, srv2012561161=51, srv1805310755=37, srv1390020735=14, srv659166197=84} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-24 13:53:36,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-24 13:53:36,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-24 13:53:36,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-24 13:53:36,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-24 13:53:36,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-24 13:53:36,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-24 13:53:36,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-24 13:53:36,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-24 13:53:36,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-24 13:53:36,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-24 13:53:36,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-24 13:53:36,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-24 13:53:36,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-24 13:53:36,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-24 13:53:36,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-24 13:53:36,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-24 13:53:36,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-24 13:53:36,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-24 13:53:36,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-24 13:53:36,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-24 13:53:36,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-24 13:53:36,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-24 13:53:36,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-24 13:53:36,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-24 13:53:36,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-24 13:53:36,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-24 13:53:36,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-24 13:53:36,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-24 13:53:36,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-24 13:53:36,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-24 13:53:36,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-24 13:53:36,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-24 13:53:36,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-24 13:53:36,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-24 13:53:36,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-24 13:53:36,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-24 13:53:36,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-24 13:53:36,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-24 13:53:36,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-24 13:53:36,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-24 13:53:36,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-24 13:53:36,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-24 13:53:36,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-24 13:53:36,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-24 13:53:36,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-24 13:53:36,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-24 13:53:36,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-24 13:53:36,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-24 13:53:36,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-24 13:53:36,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-24 13:53:36,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-24 13:53:36,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-24 13:53:36,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-24 13:53:36,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-24 13:53:36,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-24 13:53:36,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-24 13:53:36,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-24 13:53:36,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-24 13:53:36,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-24 13:53:36,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-24 13:53:36,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-24 13:53:36,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-24 13:53:36,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-24 13:53:36,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-24 13:53:36,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-24 13:53:36,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-24 13:53:36,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-24 13:53:36,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-24 13:53:36,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-24 13:53:36,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-24 13:53:36,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-24 13:53:36,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-24 13:53:36,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-24 13:53:36,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-24 13:53:36,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-24 13:53:36,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-24 13:53:36,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-24 13:53:36,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-24 13:53:36,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-24 13:53:36,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-24 13:53:36,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-24 13:53:36,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-24 13:53:36,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-24 13:53:36,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-24 13:53:36,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-24 13:53:36,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-24 13:53:36,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-24 13:53:36,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-24 13:53:36,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-24 13:53:36,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-24 13:53:36,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-24 13:53:36,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-24 13:53:36,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-24 13:53:36,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-24 13:53:36,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-24 13:53:36,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-24 13:53:36,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-24 13:53:36,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-24 13:53:36,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-24 13:53:36,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-24 13:53:36,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-24 13:53:36,225 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006104220173631566); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0113212452865072); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006104220173631566); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-24 13:53:36,225 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-24 13:53:36,226 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=6.655417452411074E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006104220173631566); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0113212452865072); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006104220173631566); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-24 13:53:36,466 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 334 ms to try 5000 different iterations.  Found a solution that moves 256 regions; Going from a computed imbalance of 6.655417452411074E-4 to a new imbalance of 4.3980232316624037E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0021711298242630987); MoveCostFunction : (multiplier=7.0, imbalance=0.028444444444444446, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-24 13:53:36,467 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv997980367:82 , srv1395364996:83 , srv1440059620:83 , srv1686719182:83 , srv191837390:83 , srv2012561161:83 , srv43702921:83 , srv542874362:83 , srv562153112:83 , srv576942190:83 , srv655815372:83 , srv916587316:83 , srv1007938144:84 , srv1049888292:84 , srv1177536125:84 , srv1396174741:84 , srv1435682949:84 , srv1451689622:84 , srv166108406:84 , srv1718635216:84 , srv172732176:84 , srv1827453345:84 , srv1851587644:84 , srv1874348089:84 , srv1970667114:84 , srv1987567038:84 , srv2082522020:84 , srv2118787632:84 , srv2137861069:84 , srv2139824659:84 , srv286122172:84 , srv384923833:84 , srv393923538:84 , srv44095414:84 , srv536469789:84 , srv581594779:84 , srv59926084:84 , srv832034497:84 , srv882082258:84 , srv914233195:84 , srv961248162:84 , srv1085207044:85 , srv111174730:85 , srv1112756844:85 , srv1124506456:85 , srv1135719273:85 , srv122073948:85 , srv1275946718:85 , srv1295387511:85 , srv1296423549:85 , srv1382981855:85 , srv1390020735:85 , srv1484810033:85 , srv151452705:85 , srv1560863829:85 , srv1563407380:85 , srv1563577225:85 , srv1577149768:85 , srv16256222:85 , srv1642580980:85 , srv1654143018:85 , srv1696198163:85 , srv176484889:85 , srv1805310755:85 , srv1850933261:85 , srv1881815539:85 , srv1917560528:85 , srv2010836640:85 , srv2067681382:85 , srv2090454543:85 , srv210298266:85 , srv213477239:85 , srv219967186:85 , srv243150057:85 , srv356094770:85 , srv362015264:85 , srv370784493:85 , srv440275323:85 , srv451274144:85 , srv52057865:85 , srv652772091:85 , srv659166197:85 , srv671329866:85 , srv76372558:85 , srv812999447:85 , srv871001218:85 , srv891802918:85 , srv963171803:85 , srv963450734:85 , srv970174018:85 , srv1349217485:86 , srv1618243627:86 , srv1760441402:86 , srv1810158322:86 , srv1825463833:86 , srv1850575555:86 , srv2137040705:86 , srv338810102:86 , srv721886421:86 , srv528017155:630 }
2023-04-24 13:53:36,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-24 13:53:36,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv111174730=3, srv1881815539=45, srv970174018=98, srv1760441402=35, srv2067681382=52, srv1851587644=43, srv1295387511=10, srv581594779=80, srv871001218=90, srv1642580980=28, srv914233195=93, srv1560863829=22, srv2082522020=53, srv542874362=77, srv963450734=97, srv1112756844=4, srv1435682949=17, srv2137040705=58, srv891802918=92, srv76372558=87, srv671329866=85, srv243150057=62, srv362015264=66, srv1577149768=25, srv1696198163=32, srv562153112=78, srv576942190=79, srv2137861069=59, srv2010836640=50, srv1395364996=15, srv882082258=91, srv1349217485=12, srv210298266=55, srv338810102=64, srv1007938144=0, srv356094770=65, srv1850575555=41, srv1440059620=18, srv1827453345=40, srv172732176=34, srv721886421=86, srv1124506456=5, srv2118787632=56, srv1850933261=42, srv384923833=68, srv1654143018=29, srv43702921=70, srv832034497=89, srv963171803=96, srv440275323=71, srv1135719273=6, srv166108406=30, srv536469789=76, srv370784493=67, srv1451689622=19, srv812999447=88, srv1296423549=11, srv1810158322=38, srv1917560528=46, srv2090454543=54, srv59926084=81, srv1484810033=20, srv1686719182=31, srv1396174741=16, srv655815372=83, srv44095414=72, srv2139824659=60, srv997980367=99, srv219967186=61, srv393923538=69, srv1177536125=7, srv1049888292=1, srv1085207044=2, srv1718635216=33, srv916587316=94, srv286122172=63, srv176484889=36, srv1825463833=39, srv451274144=73, srv961248162=95, srv1970667114=48, srv1382981855=13, srv52057865=74, srv151452705=21, srv652772091=82, srv1563577225=24, srv1987567038=49, srv122073948=8, srv1563407380=23, srv1874348089=44, srv191837390=47, srv16256222=27, srv528017155=75, srv1275946718=9, srv1618243627=26, srv213477239=57, srv2012561161=51, srv1805310755=37, srv1390020735=14, srv659166197=84} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-24 13:53:36,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-24 13:53:36,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-24 13:53:36,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-24 13:53:36,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-24 13:53:36,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-24 13:53:36,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-24 13:53:36,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-24 13:53:36,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-24 13:53:36,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-24 13:53:36,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-24 13:53:36,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-24 13:53:36,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-24 13:53:36,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-24 13:53:36,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-24 13:53:36,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-24 13:53:36,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-24 13:53:36,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-24 13:53:36,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-24 13:53:36,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-24 13:53:36,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-24 13:53:36,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-24 13:53:36,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-24 13:53:36,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-24 13:53:36,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-24 13:53:36,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-24 13:53:36,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-24 13:53:36,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-24 13:53:36,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-24 13:53:36,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-24 13:53:36,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-24 13:53:36,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-24 13:53:36,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-24 13:53:36,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-24 13:53:36,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-24 13:53:36,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-24 13:53:36,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-24 13:53:36,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-24 13:53:36,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-24 13:53:36,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-24 13:53:36,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-24 13:53:36,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-24 13:53:36,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-24 13:53:36,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-24 13:53:36,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-24 13:53:36,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-24 13:53:36,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-24 13:53:36,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-24 13:53:36,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-24 13:53:36,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-24 13:53:36,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-24 13:53:36,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-24 13:53:36,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-24 13:53:36,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-24 13:53:36,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-24 13:53:36,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-24 13:53:36,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-24 13:53:36,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-24 13:53:36,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-24 13:53:36,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-24 13:53:36,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-24 13:53:36,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-24 13:53:36,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-24 13:53:36,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-24 13:53:36,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-24 13:53:36,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-24 13:53:36,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-24 13:53:36,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-24 13:53:36,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-24 13:53:36,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-24 13:53:36,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-24 13:53:36,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-24 13:53:36,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-24 13:53:36,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-24 13:53:36,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-24 13:53:36,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-24 13:53:36,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-24 13:53:36,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-24 13:53:36,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-24 13:53:36,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-24 13:53:36,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-24 13:53:36,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-24 13:53:36,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-24 13:53:36,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-24 13:53:36,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-24 13:53:36,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-24 13:53:36,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-24 13:53:36,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-24 13:53:36,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-24 13:53:36,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-24 13:53:36,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-24 13:53:36,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-24 13:53:36,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-24 13:53:36,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-24 13:53:36,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-24 13:53:36,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-24 13:53:36,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-24 13:53:36,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-24 13:53:36,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-24 13:53:36,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-24 13:53:36,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-24 13:53:36,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-24 13:53:36,574 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0021711298242630987); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-24 13:53:36,575 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-24 13:53:36,575 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=4.2187682003677966E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0021711298242630987); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-24 13:53:36,745 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 248 ms to try 5000 different iterations.  Found a solution that moves 33 regions; Going from a computed imbalance of 4.2187682003677966E-5 to a new imbalance of 7.920492739128986E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017082358106351274); MoveCostFunction : (multiplier=7.0, imbalance=0.0036666666666666666); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-24 13:53:36,746 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1382981855:83 , srv1563577225:83 , srv1618243627:83 , srv1696198163:83 , srv1810158322:83 , srv1850575555:83 , srv1917560528:83 , srv2137040705:83 , srv721886421:83 , srv970174018:83 , srv1007938144:84 , srv1049888292:84 , srv1085207044:84 , srv111174730:84 , srv1112756844:84 , srv1124506456:84 , srv1135719273:84 , srv1177536125:84 , srv122073948:84 , srv1275946718:84 , srv1295387511:84 , srv1296423549:84 , srv1349217485:84 , srv1390020735:84 , srv1396174741:84 , srv1435682949:84 , srv1440059620:84 , srv1451689622:84 , srv1484810033:84 , srv151452705:84 , srv1560863829:84 , srv1563407380:84 , srv1577149768:84 , srv16256222:84 , srv1642580980:84 , srv1654143018:84 , srv166108406:84 , srv1718635216:84 , srv172732176:84 , srv1760441402:84 , srv176484889:84 , srv1805310755:84 , srv1825463833:84 , srv1827453345:84 , srv1850933261:84 , srv1851587644:84 , srv1874348089:84 , srv1881815539:84 , srv1970667114:84 , srv1987567038:84 , srv2010836640:84 , srv2067681382:84 , srv2082522020:84 , srv2090454543:84 , srv210298266:84 , srv2118787632:84 , srv213477239:84 , srv2137861069:84 , srv2139824659:84 , srv219967186:84 , srv243150057:84 , srv286122172:84 , srv338810102:84 , srv356094770:84 , srv362015264:84 , srv370784493:84 , srv384923833:84 , srv393923538:84 , srv440275323:84 , srv451274144:84 , srv52057865:84 , srv536469789:84 , srv542874362:84 , srv562153112:84 , srv581594779:84 , srv59926084:84 , srv652772091:84 , srv659166197:84 , srv671329866:84 , srv76372558:84 , srv812999447:84 , srv832034497:84 , srv871001218:84 , srv882082258:84 , srv891802918:84 , srv916587316:84 , srv961248162:84 , srv963171803:84 , srv963450734:84 , srv1395364996:85 , srv1686719182:85 , srv191837390:85 , srv2012561161:85 , srv43702921:85 , srv44095414:85 , srv576942190:85 , srv655815372:85 , srv914233195:85 , srv997980367:85 , srv528017155:684 }
2023-04-24 13:53:36,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-24 13:53:36,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv111174730=3, srv1881815539=45, srv970174018=98, srv1760441402=35, srv2067681382=52, srv1851587644=43, srv1295387511=10, srv581594779=80, srv871001218=90, srv1642580980=28, srv914233195=93, srv1560863829=22, srv2082522020=53, srv542874362=77, srv963450734=97, srv1112756844=4, srv1435682949=17, srv2137040705=58, srv891802918=92, srv76372558=87, srv671329866=85, srv243150057=62, srv362015264=66, srv1577149768=25, srv1696198163=32, srv562153112=78, srv576942190=79, srv2137861069=59, srv2010836640=50, srv1395364996=15, srv882082258=91, srv1349217485=12, srv210298266=55, srv338810102=64, srv1007938144=0, srv356094770=65, srv1850575555=41, srv1440059620=18, srv1827453345=40, srv172732176=34, srv721886421=86, srv1124506456=5, srv2118787632=56, srv1850933261=42, srv384923833=68, srv1654143018=29, srv43702921=70, srv832034497=89, srv963171803=96, srv440275323=71, srv1135719273=6, srv166108406=30, srv536469789=76, srv370784493=67, srv1451689622=19, srv812999447=88, srv1296423549=11, srv1810158322=38, srv1917560528=46, srv2090454543=54, srv59926084=81, srv1484810033=20, srv1686719182=31, srv1396174741=16, srv655815372=83, srv44095414=72, srv2139824659=60, srv997980367=99, srv219967186=61, srv393923538=69, srv1177536125=7, srv1049888292=1, srv1085207044=2, srv1718635216=33, srv916587316=94, srv286122172=63, srv176484889=36, srv1825463833=39, srv451274144=73, srv961248162=95, srv1970667114=48, srv1382981855=13, srv52057865=74, srv151452705=21, srv652772091=82, srv1563577225=24, srv1987567038=49, srv122073948=8, srv1563407380=23, srv1874348089=44, srv191837390=47, srv16256222=27, srv528017155=75, srv1275946718=9, srv1618243627=26, srv213477239=57, srv2012561161=51, srv1805310755=37, srv1390020735=14, srv659166197=84} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-24 13:53:36,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-24 13:53:36,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-24 13:53:36,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-24 13:53:36,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-24 13:53:36,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-24 13:53:36,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-24 13:53:36,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-24 13:53:36,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-24 13:53:36,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-24 13:53:36,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-24 13:53:36,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-24 13:53:36,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-24 13:53:36,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-24 13:53:36,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-24 13:53:36,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-24 13:53:36,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-24 13:53:36,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-24 13:53:36,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-24 13:53:36,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-24 13:53:36,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-24 13:53:36,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-24 13:53:36,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-24 13:53:36,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-24 13:53:36,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-24 13:53:36,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-24 13:53:36,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-24 13:53:36,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-24 13:53:36,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-24 13:53:36,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-24 13:53:36,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-24 13:53:36,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-24 13:53:36,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-24 13:53:36,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-24 13:53:36,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-24 13:53:36,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-24 13:53:36,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-24 13:53:36,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-24 13:53:36,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-24 13:53:36,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-24 13:53:36,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-24 13:53:36,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-24 13:53:36,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-24 13:53:36,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-24 13:53:36,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-24 13:53:36,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-24 13:53:36,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-24 13:53:36,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-24 13:53:36,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-24 13:53:36,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-24 13:53:36,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-24 13:53:36,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-24 13:53:36,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-24 13:53:36,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-24 13:53:36,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-24 13:53:36,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-24 13:53:36,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-24 13:53:36,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-24 13:53:36,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-24 13:53:36,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-24 13:53:36,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-24 13:53:36,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-24 13:53:36,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-24 13:53:36,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-24 13:53:36,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-24 13:53:36,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-24 13:53:36,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-24 13:53:36,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-24 13:53:36,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-24 13:53:36,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-24 13:53:36,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-24 13:53:36,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-24 13:53:36,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-24 13:53:36,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-24 13:53:36,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-24 13:53:36,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-24 13:53:36,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-24 13:53:36,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-24 13:53:36,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-24 13:53:36,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-24 13:53:36,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-24 13:53:36,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-24 13:53:36,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-24 13:53:36,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-24 13:53:36,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-24 13:53:36,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-24 13:53:36,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-24 13:53:36,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-24 13:53:36,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-24 13:53:36,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-24 13:53:36,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-24 13:53:36,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-24 13:53:36,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-24 13:53:36,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-24 13:53:36,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-24 13:53:36,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-24 13:53:36,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-24 13:53:36,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-24 13:53:36,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-24 13:53:36,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-24 13:53:36,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-24 13:53:36,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-24 13:53:36,850 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=7.689421800350782E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017082358106351274); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-24 13:53:36,850 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1382981855:83 , srv1563577225:83 , srv1618243627:83 , srv1696198163:83 , srv1810158322:83 , srv1850575555:83 , srv1917560528:83 , srv2137040705:83 , srv721886421:83 , srv970174018:83 , srv1007938144:84 , srv1049888292:84 , srv1085207044:84 , srv111174730:84 , srv1112756844:84 , srv1124506456:84 , srv1135719273:84 , srv1177536125:84 , srv122073948:84 , srv1275946718:84 , srv1295387511:84 , srv1296423549:84 , srv1349217485:84 , srv1390020735:84 , srv1396174741:84 , srv1435682949:84 , srv1440059620:84 , srv1451689622:84 , srv1484810033:84 , srv151452705:84 , srv1560863829:84 , srv1563407380:84 , srv1577149768:84 , srv16256222:84 , srv1642580980:84 , srv1654143018:84 , srv166108406:84 , srv1718635216:84 , srv172732176:84 , srv1760441402:84 , srv176484889:84 , srv1805310755:84 , srv1825463833:84 , srv1827453345:84 , srv1850933261:84 , srv1851587644:84 , srv1874348089:84 , srv1881815539:84 , srv1970667114:84 , srv1987567038:84 , srv2010836640:84 , srv2067681382:84 , srv2082522020:84 , srv2090454543:84 , srv210298266:84 , srv2118787632:84 , srv213477239:84 , srv2137861069:84 , srv2139824659:84 , srv219967186:84 , srv243150057:84 , srv286122172:84 , srv338810102:84 , srv356094770:84 , srv362015264:84 , srv370784493:84 , srv384923833:84 , srv393923538:84 , srv440275323:84 , srv451274144:84 , srv52057865:84 , srv536469789:84 , srv542874362:84 , srv562153112:84 , srv581594779:84 , srv59926084:84 , srv652772091:84 , srv659166197:84 , srv671329866:84 , srv76372558:84 , srv812999447:84 , srv832034497:84 , srv871001218:84 , srv882082258:84 , srv891802918:84 , srv916587316:84 , srv961248162:84 , srv963171803:84 , srv963450734:84 , srv1395364996:85 , srv1686719182:85 , srv191837390:85 , srv2012561161:85 , srv43702921:85 , srv44095414:85 , srv576942190:85 , srv655815372:85 , srv914233195:85 , srv997980367:85 , srv528017155:684 }
2023-04-24 13:53:36,872 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=404 (was 404), ProcessCount=172 (was 170) - ProcessCount LEAK? -, AvailableMemoryMB=8508 (was 8998)
2023-04-24 13:53:36,878 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=404, ProcessCount=172, AvailableMemoryMB=8507
2023-04-24 13:53:36,878 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-24 13:53:36,878 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-04-24 13:53:36,884 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1005480060:84 , srv103134305:84 , srv1054279029:84 , srv1236090579:84 , srv1433971660:84 , srv1435862758:84 , srv145688950:84 , srv1509199371:84 , srv1661798195:84 , srv1675339768:84 , srv1848063130:84 , srv1958882635:84 , srv1976770120:84 , srv1992526024:84 , srv2130274620:84 , srv2139850528:84 , srv273474691:84 , srv280706016:84 , srv409858677:84 , srv523159853:84 , srv533653491:84 , srv663486054:84 , srv677213390:84 , srv683113209:84 , srv711562075:84 , srv841908435:84 , srv864075422:84 , srv936812951:84 , srv97132640:84 , srv1778345758:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-04-24 13:53:36,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-24 13:53:36,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1054279029=2, srv1992526024=14, srv1433971660=4, srv1661798195=8, srv864075422=27, srv1976770120=13, srv1958882635=12, srv663486054=22, srv2130274620=15, srv1005480060=0, srv677213390=23, srv683113209=24, srv841908435=26, srv1435862758=5, srv1848063130=11, srv936812951=28, srv1675339768=9, srv273474691=17, srv711562075=25, srv523159853=20, srv2139850528=16, srv280706016=18, srv145688950=6, srv97132640=29, srv103134305=1, srv1509199371=7, srv409858677=19, srv1236090579=3, srv1778345758=10, srv533653491=21} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-24 13:53:36,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-24 13:53:36,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-24 13:53:36,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-24 13:53:36,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-24 13:53:36,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-24 13:53:36,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-24 13:53:36,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-24 13:53:36,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-24 13:53:36,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-24 13:53:36,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-24 13:53:36,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-24 13:53:36,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-24 13:53:36,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-24 13:53:36,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-24 13:53:36,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-24 13:53:36,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-24 13:53:36,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-24 13:53:36,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-24 13:53:36,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-24 13:53:36,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-24 13:53:36,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-24 13:53:36,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-24 13:53:36,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-24 13:53:36,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-24 13:53:36,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-24 13:53:36,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-24 13:53:36,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-24 13:53:36,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-24 13:53:36,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-24 13:53:36,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-24 13:53:36,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-04-24 13:53:36,906 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0830971753684453, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-24 13:53:36,907 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912454879750494, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0830971753684453, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-04-24 13:55:36,897 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 9786139 different iterations.  Found a solution that moves 1839 regions; Going from a computed imbalance of 0.9912454879750494 to a new imbalance of 4.292317741546655E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6811111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-24 13:55:36,902 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1005480060:90 , srv103134305:90 , srv1054279029:90 , srv1236090579:90 , srv1433971660:90 , srv1435862758:90 , srv145688950:90 , srv1509199371:90 , srv1661798195:90 , srv1675339768:90 , srv1778345758:90 , srv1848063130:90 , srv1958882635:90 , srv1976770120:90 , srv1992526024:90 , srv2130274620:90 , srv2139850528:90 , srv273474691:90 , srv280706016:90 , srv409858677:90 , srv523159853:90 , srv533653491:90 , srv663486054:90 , srv677213390:90 , srv683113209:90 , srv711562075:90 , srv841908435:90 , srv864075422:90 , srv936812951:90 , srv97132640:90 }
2023-04-24 13:55:36,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-24 13:55:36,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1054279029=2, srv1992526024=14, srv1433971660=4, srv1661798195=8, srv864075422=27, srv1976770120=13, srv1958882635=12, srv663486054=22, srv2130274620=15, srv1005480060=0, srv677213390=23, srv683113209=24, srv841908435=26, srv1435862758=5, srv1848063130=11, srv936812951=28, srv1675339768=9, srv273474691=17, srv711562075=25, srv523159853=20, srv2139850528=16, srv280706016=18, srv145688950=6, srv97132640=29, srv103134305=1, srv1509199371=7, srv409858677=19, srv1236090579=3, srv1778345758=10, srv533653491=21} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-24 13:55:36,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-24 13:55:36,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-24 13:55:36,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-24 13:55:36,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-24 13:55:36,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-24 13:55:36,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-24 13:55:36,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-24 13:55:36,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-24 13:55:36,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-24 13:55:36,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-24 13:55:36,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-24 13:55:36,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-24 13:55:36,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-24 13:55:36,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-24 13:55:36,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-24 13:55:36,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-24 13:55:36,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-24 13:55:36,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-24 13:55:36,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-24 13:55:36,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-24 13:55:36,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-24 13:55:36,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-24 13:55:36,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-24 13:55:36,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-24 13:55:36,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-24 13:55:36,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-24 13:55:36,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-24 13:55:36,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-24 13:55:36,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-24 13:55:36,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-24 13:55:36,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-04-24 13:55:36,916 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-24 13:55:36,916 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1005480060:90 , srv103134305:90 , srv1054279029:90 , srv1236090579:90 , srv1433971660:90 , srv1435862758:90 , srv145688950:90 , srv1509199371:90 , srv1661798195:90 , srv1675339768:90 , srv1778345758:90 , srv1848063130:90 , srv1958882635:90 , srv1976770120:90 , srv1992526024:90 , srv2130274620:90 , srv2139850528:90 , srv273474691:90 , srv280706016:90 , srv409858677:90 , srv523159853:90 , srv533653491:90 , srv663486054:90 , srv677213390:90 , srv683113209:90 , srv711562075:90 , srv841908435:90 , srv864075422:90 , srv936812951:90 , srv97132640:90 }
2023-04-24 13:55:36,929 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=199 (was 404), ProcessCount=169 (was 172), AvailableMemoryMB=8629 (was 8507) - AvailableMemoryMB LEAK? -