2023-04-30 04:53:31,815 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-04-30 04:53:32,155 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-04-30 04:53:32,277 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-04-30 04:53:32,350 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-30 04:53:32,360 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-04-30 04:53:32,378 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=257, ProcessCount=170, AvailableMemoryMB=11300
2023-04-30 04:53:32,381 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-30 04:53:32,381 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-04-30 04:53:32,513 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1051457520:84 , srv1080346090:84 , srv1094275652:84 , srv1123713316:84 , srv1139774916:84 , srv1141771707:84 , srv1145695496:84 , srv1145707623:84 , srv1154047462:84 , srv1155433441:84 , srv1213681385:84 , srv1232484357:84 , srv1267740478:84 , srv1273903727:84 , srv1299801610:84 , srv1364842120:84 , srv1375163110:84 , srv1383940672:84 , srv1424658731:84 , srv1447538654:84 , srv1450596636:84 , srv1504029417:84 , srv1508232990:84 , srv1529867806:84 , srv1562651081:84 , srv1573494540:84 , srv1591220727:84 , srv1596250195:84 , srv1601485067:84 , srv1617216764:84 , srv1665697286:84 , srv1689189255:84 , srv1726182669:84 , srv1750673279:84 , srv1754415008:84 , srv1797084618:84 , srv1826637472:84 , srv1826989720:84 , srv1836853232:84 , srv1862274765:84 , srv1863107064:84 , srv1866245429:84 , srv1871415940:84 , srv1887091254:84 , srv1919705034:84 , srv1952630660:84 , srv1998654150:84 , srv2011898781:84 , srv2017744360:84 , srv2022855665:84 , srv2031402093:84 , srv2051452840:84 , srv210146527:84 , srv2110312128:84 , srv212420269:84 , srv2143721424:84 , srv223194128:84 , srv223689801:84 , srv229131794:84 , srv257020511:84 , srv281454017:84 , srv289295242:84 , srv386524453:84 , srv397170164:84 , srv42483158:84 , srv42899276:84 , srv430243969:84 , srv449906673:84 , srv496802350:84 , srv527020219:84 , srv528305636:84 , srv528787434:84 , srv538905501:84 , srv539629846:84 , srv54488484:84 , srv546531135:84 , srv548630484:84 , srv55450174:84 , srv563421758:84 , srv600086848:84 , srv616886713:84 , srv622216481:84 , srv691836265:84 , srv694137785:84 , srv705700741:84 , srv74630509:84 , srv751782089:84 , srv778955060:84 , srv799557335:84 , srv81083320:84 , srv820409871:84 , srv835740405:84 , srv840346632:84 , srv849667549:84 , srv852644063:84 , srv85907115:84 , srv867559295:84 , srv940339403:84 , srv941432090:84 , srv1149744427:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-04-30 04:53:32,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-30 04:53:32,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1998654150=47, srv1145695496=6, srv1267740478=13, srv1299801610=15, srv289295242=62, srv546531135=76, srv548630484=77, srv799557335=89, srv430243969=67, srv281454017=61, srv2017744360=49, srv538905501=73, srv1139774916=4, srv1573494540=26, srv705700741=85, srv397170164=64, srv1080346090=1, srv1123713316=3, srv1862274765=40, srv74630509=86, srv751782089=87, srv1726182669=33, srv55450174=78, srv849667549=94, srv1051457520=0, srv528305636=71, srv1866245429=42, srv2022855665=50, srv1750673279=34, srv528787434=72, srv1213681385=11, srv2031402093=51, srv42899276=66, srv867559295=97, srv1826637472=37, srv210146527=53, srv1145707623=7, srv1665697286=31, srv1887091254=44, srv42483158=65, srv1149744427=8, srv1094275652=2, srv496802350=69, srv539629846=74, srv1863107064=41, srv616886713=81, srv449906673=68, srv1826989720=38, srv563421758=79, srv1273903727=14, srv1383940672=18, srv1529867806=24, srv600086848=80, srv1364842120=16, srv820409871=91, srv1617216764=30, srv1689189255=32, srv2110312128=54, srv1375163110=17, srv1754415008=35, srv1562651081=25, srv527020219=70, srv2143721424=56, srv257020511=60, srv229131794=59, srv1836853232=39, srv54488484=75, srv941432090=99, srv694137785=84, srv835740405=92, srv85907115=96, srv2051452840=52, srv1508232990=23, srv622216481=82, srv1601485067=29, srv1450596636=21, srv81083320=90, srv223689801=58, srv1871415940=43, srv691836265=83, srv1424658731=19, srv1504029417=22, srv1447538654=20, srv1919705034=45, srv212420269=55, srv840346632=93, srv1591220727=27, srv2011898781=48, srv940339403=98, srv223194128=57, srv778955060=88, srv1232484357=12, srv1154047462=9, srv852644063=95, srv1155433441=10, srv386524453=63, srv1141771707=5, srv1952630660=46, srv1797084618=36, srv1596250195=28} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-30 04:53:32,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-30 04:53:32,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-30 04:53:32,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-30 04:53:32,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-30 04:53:32,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-30 04:53:32,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-30 04:53:32,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-30 04:53:32,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-30 04:53:32,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-30 04:53:32,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-30 04:53:32,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-30 04:53:32,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-30 04:53:32,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-30 04:53:32,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-30 04:53:32,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-30 04:53:32,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-30 04:53:32,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-30 04:53:32,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-30 04:53:32,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-30 04:53:32,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-30 04:53:32,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-30 04:53:32,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-30 04:53:32,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-30 04:53:32,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-30 04:53:32,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-30 04:53:32,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-30 04:53:32,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-30 04:53:32,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-30 04:53:32,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-30 04:53:32,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-30 04:53:32,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-30 04:53:32,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-30 04:53:32,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-30 04:53:32,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-30 04:53:32,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-30 04:53:32,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-30 04:53:32,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-30 04:53:32,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-30 04:53:32,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-30 04:53:32,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-30 04:53:32,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-30 04:53:32,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-30 04:53:32,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-30 04:53:32,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-30 04:53:32,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-30 04:53:32,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-30 04:53:32,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-30 04:53:32,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-30 04:53:32,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-30 04:53:32,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-30 04:53:32,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-30 04:53:32,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-30 04:53:32,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-30 04:53:32,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-30 04:53:32,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-30 04:53:32,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-30 04:53:32,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-30 04:53:32,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-30 04:53:32,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-30 04:53:32,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-30 04:53:32,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-30 04:53:32,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-30 04:53:32,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-30 04:53:32,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-30 04:53:32,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-30 04:53:32,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-30 04:53:32,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-30 04:53:32,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-30 04:53:32,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-30 04:53:32,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-30 04:53:32,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-30 04:53:32,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-30 04:53:32,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-30 04:53:32,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-30 04:53:32,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-30 04:53:32,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-30 04:53:32,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-30 04:53:32,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-30 04:53:32,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-30 04:53:32,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-30 04:53:32,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-30 04:53:32,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-30 04:53:32,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-30 04:53:32,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-30 04:53:32,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-30 04:53:32,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-30 04:53:32,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-30 04:53:32,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-30 04:53:32,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-30 04:53:32,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-30 04:53:32,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-30 04:53:32,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-30 04:53:32,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-30 04:53:32,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-30 04:53:32,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-30 04:53:32,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-30 04:53:32,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-30 04:53:32,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-30 04:53:32,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-30 04:53:32,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-30 04:53:32,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-30 04:53:32,902 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-30 04:53:32,902 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-30 04:53:32,903 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-30 04:53:33,349 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 689 ms to try 5000 different iterations.  Found a solution that moves 4672 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.183134909041494. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04276483982347906, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05779331570404435, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5191111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04276483982347906, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17858333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24283333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-30 04:53:33,372 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1141771707:80 , srv1617216764:80 , srv528305636:80 , srv840346632:80 , srv1364842120:81 , srv1573494540:81 , srv289295242:81 , srv600086848:81 , srv1375163110:82 , srv622216481:82 , srv81083320:82 , srv1145707623:83 , srv1267740478:83 , srv1450596636:83 , srv1866245429:83 , srv1094275652:84 , srv1139774916:84 , srv1862274765:84 , srv1998654150:84 , srv2017744360:84 , srv212420269:84 , srv386524453:84 , srv430243969:84 , srv74630509:84 , srv940339403:84 , srv1145695496:85 , srv1154047462:85 , srv1299801610:85 , srv1424658731:85 , srv1504029417:85 , srv1529867806:85 , srv210146527:85 , srv223194128:85 , srv281454017:85 , srv42483158:85 , srv799557335:85 , srv849667549:85 , srv1123713316:86 , srv1383940672:86 , srv1596250195:86 , srv1754415008:86 , srv1797084618:86 , srv1836853232:86 , srv1952630660:86 , srv2051452840:86 , srv496802350:86 , srv527020219:86 , srv54488484:86 , srv546531135:86 , srv563421758:86 , srv691836265:86 , srv820409871:86 , srv941432090:86 , srv1051457520:87 , srv1155433441:87 , srv1591220727:87 , srv1689189255:87 , srv1826637472:87 , srv1919705034:87 , srv2022855665:87 , srv2110312128:87 , srv229131794:87 , srv397170164:87 , srv538905501:87 , srv55450174:87 , srv616886713:87 , srv705700741:87 , srv1213681385:88 , srv1508232990:88 , srv1562651081:88 , srv1665697286:88 , srv1726182669:88 , srv1887091254:88 , srv2011898781:88 , srv2031402093:88 , srv2143721424:88 , srv42899276:88 , srv539629846:88 , srv548630484:88 , srv694137785:88 , srv751782089:88 , srv835740405:88 , srv852644063:88 , srv1447538654:89 , srv1601485067:89 , srv1826989720:89 , srv528787434:89 , srv867559295:89 , srv1080346090:90 , srv1750673279:90 , srv1871415940:90 , srv449906673:90 , srv778955060:90 , srv85907115:90 , srv1232484357:91 , srv1863107064:91 , srv223689801:92 , srv257020511:93 , srv1273903727:94 , srv1149744427:470 }
2023-04-30 04:53:33,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-30 04:53:33,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1998654150=47, srv1145695496=6, srv1267740478=13, srv1299801610=15, srv289295242=62, srv546531135=76, srv548630484=77, srv799557335=89, srv430243969=67, srv281454017=61, srv2017744360=49, srv538905501=73, srv1139774916=4, srv1573494540=26, srv705700741=85, srv397170164=64, srv1080346090=1, srv1123713316=3, srv1862274765=40, srv74630509=86, srv751782089=87, srv1726182669=33, srv55450174=78, srv849667549=94, srv1051457520=0, srv528305636=71, srv1866245429=42, srv2022855665=50, srv1750673279=34, srv528787434=72, srv1213681385=11, srv2031402093=51, srv42899276=66, srv867559295=97, srv1826637472=37, srv210146527=53, srv1145707623=7, srv1665697286=31, srv1887091254=44, srv42483158=65, srv1149744427=8, srv1094275652=2, srv496802350=69, srv539629846=74, srv1863107064=41, srv616886713=81, srv449906673=68, srv1826989720=38, srv563421758=79, srv1273903727=14, srv1383940672=18, srv1529867806=24, srv600086848=80, srv1364842120=16, srv820409871=91, srv1617216764=30, srv1689189255=32, srv2110312128=54, srv1375163110=17, srv1754415008=35, srv1562651081=25, srv527020219=70, srv2143721424=56, srv257020511=60, srv229131794=59, srv1836853232=39, srv54488484=75, srv941432090=99, srv694137785=84, srv835740405=92, srv85907115=96, srv2051452840=52, srv1508232990=23, srv622216481=82, srv1601485067=29, srv1450596636=21, srv81083320=90, srv223689801=58, srv1871415940=43, srv691836265=83, srv1424658731=19, srv1504029417=22, srv1447538654=20, srv1919705034=45, srv212420269=55, srv840346632=93, srv1591220727=27, srv2011898781=48, srv940339403=98, srv223194128=57, srv778955060=88, srv1232484357=12, srv1154047462=9, srv852644063=95, srv1155433441=10, srv386524453=63, srv1141771707=5, srv1952630660=46, srv1797084618=36, srv1596250195=28} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-30 04:53:33,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-30 04:53:33,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-30 04:53:33,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-30 04:53:33,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-30 04:53:33,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-30 04:53:33,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-30 04:53:33,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-30 04:53:33,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-30 04:53:33,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-30 04:53:33,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-30 04:53:33,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-30 04:53:33,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-30 04:53:33,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-30 04:53:33,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-30 04:53:33,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-30 04:53:33,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-30 04:53:33,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-30 04:53:33,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-30 04:53:33,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-30 04:53:33,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-30 04:53:33,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-30 04:53:33,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-30 04:53:33,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-30 04:53:33,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-30 04:53:33,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-30 04:53:33,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-30 04:53:33,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-30 04:53:33,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-30 04:53:33,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-30 04:53:33,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-30 04:53:33,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-30 04:53:33,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-30 04:53:33,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-30 04:53:33,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-30 04:53:33,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-30 04:53:33,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-30 04:53:33,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-30 04:53:33,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-30 04:53:33,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-30 04:53:33,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-30 04:53:33,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-30 04:53:33,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-30 04:53:33,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-30 04:53:33,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-30 04:53:33,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-30 04:53:33,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-30 04:53:33,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-30 04:53:33,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-30 04:53:33,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-30 04:53:33,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-30 04:53:33,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-30 04:53:33,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-30 04:53:33,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-30 04:53:33,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-30 04:53:33,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-30 04:53:33,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-30 04:53:33,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-30 04:53:33,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-30 04:53:33,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-30 04:53:33,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-30 04:53:33,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-30 04:53:33,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-30 04:53:33,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-30 04:53:33,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-30 04:53:33,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-30 04:53:33,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-30 04:53:33,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-30 04:53:33,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-30 04:53:33,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-30 04:53:33,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-30 04:53:33,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-30 04:53:33,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-30 04:53:33,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-30 04:53:33,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-30 04:53:33,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-30 04:53:33,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-30 04:53:33,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-30 04:53:33,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-30 04:53:33,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-30 04:53:33,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-30 04:53:33,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-30 04:53:33,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-30 04:53:33,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-30 04:53:33,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-30 04:53:33,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-30 04:53:33,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-30 04:53:33,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-30 04:53:33,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-30 04:53:33,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-30 04:53:33,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-30 04:53:33,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-30 04:53:33,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-30 04:53:33,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-30 04:53:33,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-30 04:53:33,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-30 04:53:33,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-30 04:53:33,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-30 04:53:33,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-30 04:53:33,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-30 04:53:33,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-30 04:53:33,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-30 04:53:33,616 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04276483982347906, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05779331570404435, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04276483982347906, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17858333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24283333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-30 04:53:33,616 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-30 04:53:33,617 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18310219499828273, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04276483982347906, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05779331570404435, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04276483982347906, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17858333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24283333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-30 04:53:34,012 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 586 ms to try 5000 different iterations.  Found a solution that moves 2610 regions; Going from a computed imbalance of 0.18310219499828273 to a new imbalance of 0.01631140478771934. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02271397540915376); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03527986552320711, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02271397540915376); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013416666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043833333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-30 04:53:34,034 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv257020511:78 , srv1232484357:80 , srv1273903727:80 , srv1863107064:80 , srv1871415940:80 , srv223689801:80 , srv449906673:81 , srv1213681385:82 , srv1601485067:82 , srv1750673279:82 , srv1826989720:82 , srv528787434:82 , srv867559295:82 , srv1080346090:83 , srv1155433441:83 , srv1447538654:83 , srv539629846:83 , srv694137785:83 , srv778955060:83 , srv85907115:83 , srv1508232990:84 , srv1562651081:84 , srv1726182669:84 , srv1754415008:84 , srv1887091254:84 , srv2031402093:84 , srv2051452840:84 , srv229131794:84 , srv397170164:84 , srv548630484:84 , srv751782089:84 , srv852644063:84 , srv941432090:84 , srv1051457520:85 , srv1665697286:85 , srv1689189255:85 , srv1826637472:85 , srv281454017:85 , srv42899276:85 , srv496802350:85 , srv546531135:85 , srv820409871:85 , srv835740405:85 , srv1383940672:86 , srv1424658731:86 , srv1529867806:86 , srv1591220727:86 , srv1596250195:86 , srv1919705034:86 , srv2022855665:86 , srv2110312128:86 , srv2143721424:86 , srv223194128:86 , srv430243969:86 , srv54488484:86 , srv616886713:86 , srv799557335:86 , srv849667549:86 , srv1145695496:87 , srv1154047462:87 , srv1504029417:87 , srv1797084618:87 , srv1836853232:87 , srv1866245429:87 , srv1998654150:87 , srv210146527:87 , srv386524453:87 , srv55450174:87 , srv563421758:87 , srv705700741:87 , srv1123713316:88 , srv1139774916:88 , srv1145707623:88 , srv1267740478:88 , srv1299801610:88 , srv1375163110:88 , srv1450596636:88 , srv1862274765:88 , srv2011898781:88 , srv212420269:88 , srv42483158:88 , srv527020219:88 , srv538905501:88 , srv691836265:88 , srv74630509:88 , srv1094275652:89 , srv1573494540:89 , srv1952630660:89 , srv940339403:89 , srv2017744360:90 , srv600086848:90 , srv622216481:90 , srv81083320:90 , srv1364842120:91 , srv289295242:91 , srv528305636:91 , srv1617216764:92 , srv840346632:92 , srv1141771707:93 , srv1149744427:506 }
2023-04-30 04:53:34,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-30 04:53:34,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1998654150=47, srv1145695496=6, srv1267740478=13, srv1299801610=15, srv289295242=62, srv546531135=76, srv548630484=77, srv799557335=89, srv430243969=67, srv281454017=61, srv2017744360=49, srv538905501=73, srv1139774916=4, srv1573494540=26, srv705700741=85, srv397170164=64, srv1080346090=1, srv1123713316=3, srv1862274765=40, srv74630509=86, srv751782089=87, srv1726182669=33, srv55450174=78, srv849667549=94, srv1051457520=0, srv528305636=71, srv1866245429=42, srv2022855665=50, srv1750673279=34, srv528787434=72, srv1213681385=11, srv2031402093=51, srv42899276=66, srv867559295=97, srv1826637472=37, srv210146527=53, srv1145707623=7, srv1665697286=31, srv1887091254=44, srv42483158=65, srv1149744427=8, srv1094275652=2, srv496802350=69, srv539629846=74, srv1863107064=41, srv616886713=81, srv449906673=68, srv1826989720=38, srv563421758=79, srv1273903727=14, srv1383940672=18, srv1529867806=24, srv600086848=80, srv1364842120=16, srv820409871=91, srv1617216764=30, srv1689189255=32, srv2110312128=54, srv1375163110=17, srv1754415008=35, srv1562651081=25, srv527020219=70, srv2143721424=56, srv257020511=60, srv229131794=59, srv1836853232=39, srv54488484=75, srv941432090=99, srv694137785=84, srv835740405=92, srv85907115=96, srv2051452840=52, srv1508232990=23, srv622216481=82, srv1601485067=29, srv1450596636=21, srv81083320=90, srv223689801=58, srv1871415940=43, srv691836265=83, srv1424658731=19, srv1504029417=22, srv1447538654=20, srv1919705034=45, srv212420269=55, srv840346632=93, srv1591220727=27, srv2011898781=48, srv940339403=98, srv223194128=57, srv778955060=88, srv1232484357=12, srv1154047462=9, srv852644063=95, srv1155433441=10, srv386524453=63, srv1141771707=5, srv1952630660=46, srv1797084618=36, srv1596250195=28} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-30 04:53:34,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-30 04:53:34,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-30 04:53:34,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-30 04:53:34,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-30 04:53:34,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-30 04:53:34,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-30 04:53:34,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-30 04:53:34,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-30 04:53:34,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-30 04:53:34,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-30 04:53:34,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-30 04:53:34,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-30 04:53:34,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-30 04:53:34,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-30 04:53:34,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-30 04:53:34,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-30 04:53:34,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-30 04:53:34,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-30 04:53:34,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-30 04:53:34,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-30 04:53:34,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-30 04:53:34,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-30 04:53:34,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-30 04:53:34,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-30 04:53:34,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-30 04:53:34,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-30 04:53:34,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-30 04:53:34,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-30 04:53:34,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-30 04:53:34,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-30 04:53:34,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-30 04:53:34,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-30 04:53:34,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-30 04:53:34,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-30 04:53:34,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-30 04:53:34,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-30 04:53:34,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-30 04:53:34,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-30 04:53:34,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-30 04:53:34,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-30 04:53:34,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-30 04:53:34,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-30 04:53:34,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-30 04:53:34,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-30 04:53:34,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-30 04:53:34,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-30 04:53:34,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-30 04:53:34,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-30 04:53:34,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-30 04:53:34,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-30 04:53:34,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-30 04:53:34,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-30 04:53:34,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-30 04:53:34,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-30 04:53:34,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-30 04:53:34,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-30 04:53:34,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-30 04:53:34,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-30 04:53:34,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-30 04:53:34,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-30 04:53:34,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-30 04:53:34,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-30 04:53:34,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-30 04:53:34,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-30 04:53:34,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-30 04:53:34,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-30 04:53:34,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-30 04:53:34,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-30 04:53:34,072 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-30 04:53:34,072 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-30 04:53:34,072 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-30 04:53:34,072 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-30 04:53:34,072 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-30 04:53:34,072 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-30 04:53:34,072 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-30 04:53:34,072 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-30 04:53:34,072 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-30 04:53:34,073 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-30 04:53:34,073 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-30 04:53:34,073 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-30 04:53:34,073 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-30 04:53:34,073 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-30 04:53:34,073 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-30 04:53:34,073 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-30 04:53:34,073 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-30 04:53:34,073 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-30 04:53:34,074 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-30 04:53:34,074 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-30 04:53:34,074 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-30 04:53:34,074 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-30 04:53:34,074 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-30 04:53:34,074 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-30 04:53:34,074 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-30 04:53:34,074 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-30 04:53:34,074 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-30 04:53:34,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-30 04:53:34,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-30 04:53:34,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-30 04:53:34,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-30 04:53:34,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-30 04:53:34,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-30 04:53:34,259 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02271397540915376); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03527986552320711, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02271397540915376); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013416666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043833333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-30 04:53:34,260 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-30 04:53:34,260 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.01629312917710688, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02271397540915376); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03527986552320711, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02271397540915376); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013416666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043833333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-30 04:53:34,581 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 489 ms to try 5000 different iterations.  Found a solution that moves 975 regions; Going from a computed imbalance of 0.01629312917710688 to a new imbalance of 0.001015997326781173. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004979068295114799); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008863600391305173); MoveCostFunction : (multiplier=7.0, imbalance=0.10833333333333334, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.004979068295114799); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.833333333333334E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004666666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-30 04:53:34,582 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1952630660:82 , srv2011898781:82 , srv2143721424:83 , srv55450174:83 , srv1094275652:84 , srv1123713316:84 , srv1141771707:84 , srv1273903727:84 , srv1299801610:84 , srv1797084618:84 , srv1836853232:84 , srv2017744360:84 , srv2110312128:84 , srv42899276:84 , srv527020219:84 , srv538905501:84 , srv691836265:84 , srv705700741:84 , srv778955060:84 , srv85907115:84 , srv940339403:84 , srv1051457520:85 , srv1080346090:85 , srv1139774916:85 , srv1364842120:85 , srv1383940672:85 , srv1447538654:85 , srv1562651081:85 , srv1591220727:85 , srv1596250195:85 , srv1665697286:85 , srv1689189255:85 , srv1750673279:85 , srv1919705034:85 , srv2022855665:85 , srv212420269:85 , srv42483158:85 , srv548630484:85 , srv563421758:85 , srv616886713:85 , srv751782089:85 , srv835740405:85 , srv840346632:85 , srv1145695496:86 , srv1154047462:86 , srv1267740478:86 , srv1450596636:86 , srv1504029417:86 , srv1508232990:86 , srv1529867806:86 , srv1617216764:86 , srv1726182669:86 , srv1826637472:86 , srv1862274765:86 , srv1863107064:86 , srv1887091254:86 , srv2031402093:86 , srv210146527:86 , srv229131794:86 , srv289295242:86 , srv397170164:86 , srv496802350:86 , srv528305636:86 , srv54488484:86 , srv546531135:86 , srv622216481:86 , srv694137785:86 , srv74630509:86 , srv799557335:86 , srv81083320:86 , srv820409871:86 , srv849667549:86 , srv852644063:86 , srv867559295:86 , srv1145707623:87 , srv1155433441:87 , srv1232484357:87 , srv1573494540:87 , srv1601485067:87 , srv1754415008:87 , srv1826989720:87 , srv1866245429:87 , srv1998654150:87 , srv2051452840:87 , srv223194128:87 , srv223689801:87 , srv257020511:87 , srv281454017:87 , srv386524453:87 , srv430243969:87 , srv449906673:87 , srv528787434:87 , srv539629846:87 , srv600086848:87 , srv941432090:87 , srv1213681385:88 , srv1424658731:88 , srv1871415940:88 , srv1375163110:89 , srv1149744427:526 }
2023-04-30 04:53:34,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-30 04:53:34,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1998654150=47, srv1145695496=6, srv1267740478=13, srv1299801610=15, srv289295242=62, srv546531135=76, srv548630484=77, srv799557335=89, srv430243969=67, srv281454017=61, srv2017744360=49, srv538905501=73, srv1139774916=4, srv1573494540=26, srv705700741=85, srv397170164=64, srv1080346090=1, srv1123713316=3, srv1862274765=40, srv74630509=86, srv751782089=87, srv1726182669=33, srv55450174=78, srv849667549=94, srv1051457520=0, srv528305636=71, srv1866245429=42, srv2022855665=50, srv1750673279=34, srv528787434=72, srv1213681385=11, srv2031402093=51, srv42899276=66, srv867559295=97, srv1826637472=37, srv210146527=53, srv1145707623=7, srv1665697286=31, srv1887091254=44, srv42483158=65, srv1149744427=8, srv1094275652=2, srv496802350=69, srv539629846=74, srv1863107064=41, srv616886713=81, srv449906673=68, srv1826989720=38, srv563421758=79, srv1273903727=14, srv1383940672=18, srv1529867806=24, srv600086848=80, srv1364842120=16, srv820409871=91, srv1617216764=30, srv1689189255=32, srv2110312128=54, srv1375163110=17, srv1754415008=35, srv1562651081=25, srv527020219=70, srv2143721424=56, srv257020511=60, srv229131794=59, srv1836853232=39, srv54488484=75, srv941432090=99, srv694137785=84, srv835740405=92, srv85907115=96, srv2051452840=52, srv1508232990=23, srv622216481=82, srv1601485067=29, srv1450596636=21, srv81083320=90, srv223689801=58, srv1871415940=43, srv691836265=83, srv1424658731=19, srv1504029417=22, srv1447538654=20, srv1919705034=45, srv212420269=55, srv840346632=93, srv1591220727=27, srv2011898781=48, srv940339403=98, srv223194128=57, srv778955060=88, srv1232484357=12, srv1154047462=9, srv852644063=95, srv1155433441=10, srv386524453=63, srv1141771707=5, srv1952630660=46, srv1797084618=36, srv1596250195=28} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-30 04:53:34,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-30 04:53:34,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-30 04:53:34,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-30 04:53:34,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-30 04:53:34,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-30 04:53:34,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-30 04:53:34,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-30 04:53:34,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-30 04:53:34,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-30 04:53:34,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-30 04:53:34,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-30 04:53:34,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-30 04:53:34,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-30 04:53:34,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-30 04:53:34,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-30 04:53:34,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-30 04:53:34,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-30 04:53:34,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-30 04:53:34,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-30 04:53:34,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-30 04:53:34,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-30 04:53:34,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-30 04:53:34,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-30 04:53:34,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-30 04:53:34,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-30 04:53:34,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-30 04:53:34,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-30 04:53:34,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-30 04:53:34,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-30 04:53:34,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-30 04:53:34,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-30 04:53:34,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-30 04:53:34,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-30 04:53:34,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-30 04:53:34,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-30 04:53:34,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-30 04:53:34,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-30 04:53:34,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-30 04:53:34,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-30 04:53:34,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-30 04:53:34,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-30 04:53:34,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-30 04:53:34,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-30 04:53:34,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-30 04:53:34,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-30 04:53:34,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-30 04:53:34,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-30 04:53:34,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-30 04:53:34,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-30 04:53:34,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-30 04:53:34,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-30 04:53:34,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-30 04:53:34,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-30 04:53:34,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-30 04:53:34,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-30 04:53:34,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-30 04:53:34,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-30 04:53:34,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-30 04:53:34,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-30 04:53:34,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-30 04:53:34,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-30 04:53:34,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-30 04:53:34,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-30 04:53:34,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-30 04:53:34,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-30 04:53:34,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-30 04:53:34,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-30 04:53:34,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-30 04:53:34,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-30 04:53:34,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-30 04:53:34,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-30 04:53:34,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-30 04:53:34,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-30 04:53:34,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-30 04:53:34,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-30 04:53:34,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-30 04:53:34,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-30 04:53:34,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-30 04:53:34,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-30 04:53:34,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-30 04:53:34,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-30 04:53:34,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-30 04:53:34,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-30 04:53:34,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-30 04:53:34,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-30 04:53:34,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-30 04:53:34,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-30 04:53:34,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-30 04:53:34,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-30 04:53:34,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-30 04:53:34,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-30 04:53:34,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-30 04:53:34,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-30 04:53:34,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-30 04:53:34,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-30 04:53:34,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-30 04:53:34,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-30 04:53:34,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-30 04:53:34,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-30 04:53:34,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-30 04:53:34,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-30 04:53:34,743 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004979068295114799); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008863600391305173); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.004979068295114799); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.833333333333334E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004666666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-30 04:53:34,743 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-30 04:53:34,743 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.001009170230862726, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004979068295114799); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008863600391305173); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.004979068295114799); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.833333333333334E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004666666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-30 04:53:35,132 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 506 ms to try 5000 different iterations.  Found a solution that moves 288 regions; Going from a computed imbalance of 0.001009170230862726 to a new imbalance of 4.605143222089252E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0026378918903435544); MoveCostFunction : (multiplier=7.0, imbalance=0.032, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-30 04:53:35,133 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1375163110:83 , srv1424658731:83 , srv1887091254:83 , srv257020511:83 , srv1080346090:84 , srv1139774916:84 , srv1145695496:84 , srv1145707623:84 , srv1154047462:84 , srv1213681385:84 , srv1232484357:84 , srv1267740478:84 , srv1273903727:84 , srv1504029417:84 , srv1508232990:84 , srv1591220727:84 , srv1596250195:84 , srv1601485067:84 , srv1617216764:84 , srv1826637472:84 , srv1826989720:84 , srv1862274765:84 , srv1871415940:84 , srv1919705034:84 , srv1998654150:84 , srv2011898781:84 , srv2017744360:84 , srv2022855665:84 , srv2031402093:84 , srv210146527:84 , srv2143721424:84 , srv223194128:84 , srv223689801:84 , srv289295242:84 , srv386524453:84 , srv449906673:84 , srv527020219:84 , srv528305636:84 , srv528787434:84 , srv538905501:84 , srv539629846:84 , srv54488484:84 , srv563421758:84 , srv600086848:84 , srv616886713:84 , srv622216481:84 , srv691836265:84 , srv705700741:84 , srv74630509:84 , srv778955060:84 , srv81083320:84 , srv835740405:84 , srv852644063:84 , srv1051457520:85 , srv1094275652:85 , srv1123713316:85 , srv1141771707:85 , srv1155433441:85 , srv1299801610:85 , srv1364842120:85 , srv1383940672:85 , srv1447538654:85 , srv1450596636:85 , srv1529867806:85 , srv1562651081:85 , srv1573494540:85 , srv1665697286:85 , srv1689189255:85 , srv1726182669:85 , srv1750673279:85 , srv1754415008:85 , srv1797084618:85 , srv1836853232:85 , srv1863107064:85 , srv1866245429:85 , srv1952630660:85 , srv2051452840:85 , srv2110312128:85 , srv229131794:85 , srv281454017:85 , srv397170164:85 , srv42483158:85 , srv42899276:85 , srv430243969:85 , srv546531135:85 , srv548630484:85 , srv55450174:85 , srv694137785:85 , srv751782089:85 , srv799557335:85 , srv820409871:85 , srv840346632:85 , srv849667549:85 , srv85907115:85 , srv867559295:85 , srv940339403:85 , srv941432090:85 , srv212420269:86 , srv496802350:86 , srv1149744427:640 }
2023-04-30 04:53:35,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-30 04:53:35,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1998654150=47, srv1145695496=6, srv1267740478=13, srv1299801610=15, srv289295242=62, srv546531135=76, srv548630484=77, srv799557335=89, srv430243969=67, srv281454017=61, srv2017744360=49, srv538905501=73, srv1139774916=4, srv1573494540=26, srv705700741=85, srv397170164=64, srv1080346090=1, srv1123713316=3, srv1862274765=40, srv74630509=86, srv751782089=87, srv1726182669=33, srv55450174=78, srv849667549=94, srv1051457520=0, srv528305636=71, srv1866245429=42, srv2022855665=50, srv1750673279=34, srv528787434=72, srv1213681385=11, srv2031402093=51, srv42899276=66, srv867559295=97, srv1826637472=37, srv210146527=53, srv1145707623=7, srv1665697286=31, srv1887091254=44, srv42483158=65, srv1149744427=8, srv1094275652=2, srv496802350=69, srv539629846=74, srv1863107064=41, srv616886713=81, srv449906673=68, srv1826989720=38, srv563421758=79, srv1273903727=14, srv1383940672=18, srv1529867806=24, srv600086848=80, srv1364842120=16, srv820409871=91, srv1617216764=30, srv1689189255=32, srv2110312128=54, srv1375163110=17, srv1754415008=35, srv1562651081=25, srv527020219=70, srv2143721424=56, srv257020511=60, srv229131794=59, srv1836853232=39, srv54488484=75, srv941432090=99, srv694137785=84, srv835740405=92, srv85907115=96, srv2051452840=52, srv1508232990=23, srv622216481=82, srv1601485067=29, srv1450596636=21, srv81083320=90, srv223689801=58, srv1871415940=43, srv691836265=83, srv1424658731=19, srv1504029417=22, srv1447538654=20, srv1919705034=45, srv212420269=55, srv840346632=93, srv1591220727=27, srv2011898781=48, srv940339403=98, srv223194128=57, srv778955060=88, srv1232484357=12, srv1154047462=9, srv852644063=95, srv1155433441=10, srv386524453=63, srv1141771707=5, srv1952630660=46, srv1797084618=36, srv1596250195=28} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-30 04:53:35,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-30 04:53:35,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-30 04:53:35,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-30 04:53:35,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-30 04:53:35,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-30 04:53:35,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-30 04:53:35,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-30 04:53:35,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-30 04:53:35,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-30 04:53:35,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-30 04:53:35,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-30 04:53:35,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-30 04:53:35,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-30 04:53:35,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-30 04:53:35,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-30 04:53:35,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-30 04:53:35,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-30 04:53:35,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-30 04:53:35,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-30 04:53:35,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-30 04:53:35,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-30 04:53:35,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-30 04:53:35,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-30 04:53:35,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-30 04:53:35,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-30 04:53:35,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-30 04:53:35,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-30 04:53:35,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-30 04:53:35,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-30 04:53:35,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-30 04:53:35,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-30 04:53:35,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-30 04:53:35,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-30 04:53:35,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-30 04:53:35,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-30 04:53:35,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-30 04:53:35,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-30 04:53:35,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-30 04:53:35,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-30 04:53:35,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-30 04:53:35,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-30 04:53:35,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-30 04:53:35,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-30 04:53:35,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-30 04:53:35,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-30 04:53:35,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-30 04:53:35,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-30 04:53:35,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-30 04:53:35,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-30 04:53:35,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-30 04:53:35,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-30 04:53:35,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-30 04:53:35,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-30 04:53:35,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-30 04:53:35,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-30 04:53:35,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-30 04:53:35,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-30 04:53:35,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-30 04:53:35,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-30 04:53:35,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-30 04:53:35,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-30 04:53:35,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-30 04:53:35,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-30 04:53:35,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-30 04:53:35,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-30 04:53:35,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-30 04:53:35,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-30 04:53:35,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-30 04:53:35,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-30 04:53:35,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-30 04:53:35,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-30 04:53:35,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-30 04:53:35,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-30 04:53:35,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-30 04:53:35,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-30 04:53:35,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-30 04:53:35,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-30 04:53:35,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-30 04:53:35,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-30 04:53:35,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-30 04:53:35,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-30 04:53:35,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-30 04:53:35,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-30 04:53:35,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-30 04:53:35,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-30 04:53:35,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-30 04:53:35,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-30 04:53:35,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-30 04:53:35,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-30 04:53:35,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-30 04:53:35,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-30 04:53:35,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-30 04:53:35,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-30 04:53:35,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-30 04:53:35,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-30 04:53:35,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-30 04:53:35,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-30 04:53:35,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-30 04:53:35,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-30 04:53:35,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-30 04:53:35,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-30 04:53:35,287 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0026378918903435544); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-30 04:53:35,287 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-30 04:53:35,288 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=4.403481311882819E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0026378918903435544); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-30 04:53:35,585 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 403 ms to try 5000 different iterations.  Found a solution that moves 28 regions; Going from a computed imbalance of 4.403481311882819E-5 to a new imbalance of 9.733614111988304E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002118805753879094); MoveCostFunction : (multiplier=7.0, imbalance=0.003111111111111111); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-30 04:53:35,586 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1123713316:83 , srv1665697286:83 , srv1726182669:83 , srv212420269:83 , srv223689801:83 , srv42483158:83 , srv496802350:83 , srv538905501:83 , srv1051457520:84 , srv1080346090:84 , srv1094275652:84 , srv1141771707:84 , srv1145695496:84 , srv1145707623:84 , srv1154047462:84 , srv1155433441:84 , srv1213681385:84 , srv1232484357:84 , srv1273903727:84 , srv1299801610:84 , srv1364842120:84 , srv1375163110:84 , srv1383940672:84 , srv1424658731:84 , srv1447538654:84 , srv1450596636:84 , srv1504029417:84 , srv1508232990:84 , srv1529867806:84 , srv1562651081:84 , srv1573494540:84 , srv1591220727:84 , srv1601485067:84 , srv1617216764:84 , srv1689189255:84 , srv1750673279:84 , srv1754415008:84 , srv1797084618:84 , srv1826637472:84 , srv1826989720:84 , srv1836853232:84 , srv1862274765:84 , srv1863107064:84 , srv1866245429:84 , srv1871415940:84 , srv1919705034:84 , srv1952630660:84 , srv1998654150:84 , srv2011898781:84 , srv2017744360:84 , srv2022855665:84 , srv2031402093:84 , srv2051452840:84 , srv210146527:84 , srv2110312128:84 , srv223194128:84 , srv229131794:84 , srv281454017:84 , srv289295242:84 , srv386524453:84 , srv397170164:84 , srv42899276:84 , srv430243969:84 , srv449906673:84 , srv527020219:84 , srv528787434:84 , srv539629846:84 , srv54488484:84 , srv546531135:84 , srv548630484:84 , srv55450174:84 , srv563421758:84 , srv600086848:84 , srv616886713:84 , srv622216481:84 , srv691836265:84 , srv694137785:84 , srv705700741:84 , srv74630509:84 , srv751782089:84 , srv799557335:84 , srv81083320:84 , srv820409871:84 , srv835740405:84 , srv840346632:84 , srv849667549:84 , srv852644063:84 , srv85907115:84 , srv867559295:84 , srv940339403:84 , srv941432090:84 , srv1139774916:85 , srv1267740478:85 , srv1596250195:85 , srv1887091254:85 , srv2143721424:85 , srv257020511:85 , srv528305636:85 , srv778955060:85 , srv1149744427:684 }
2023-04-30 04:53:35,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-30 04:53:35,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1998654150=47, srv1145695496=6, srv1267740478=13, srv1299801610=15, srv289295242=62, srv546531135=76, srv548630484=77, srv799557335=89, srv430243969=67, srv281454017=61, srv2017744360=49, srv538905501=73, srv1139774916=4, srv1573494540=26, srv705700741=85, srv397170164=64, srv1080346090=1, srv1123713316=3, srv1862274765=40, srv74630509=86, srv751782089=87, srv1726182669=33, srv55450174=78, srv849667549=94, srv1051457520=0, srv528305636=71, srv1866245429=42, srv2022855665=50, srv1750673279=34, srv528787434=72, srv1213681385=11, srv2031402093=51, srv42899276=66, srv867559295=97, srv1826637472=37, srv210146527=53, srv1145707623=7, srv1665697286=31, srv1887091254=44, srv42483158=65, srv1149744427=8, srv1094275652=2, srv496802350=69, srv539629846=74, srv1863107064=41, srv616886713=81, srv449906673=68, srv1826989720=38, srv563421758=79, srv1273903727=14, srv1383940672=18, srv1529867806=24, srv600086848=80, srv1364842120=16, srv820409871=91, srv1617216764=30, srv1689189255=32, srv2110312128=54, srv1375163110=17, srv1754415008=35, srv1562651081=25, srv527020219=70, srv2143721424=56, srv257020511=60, srv229131794=59, srv1836853232=39, srv54488484=75, srv941432090=99, srv694137785=84, srv835740405=92, srv85907115=96, srv2051452840=52, srv1508232990=23, srv622216481=82, srv1601485067=29, srv1450596636=21, srv81083320=90, srv223689801=58, srv1871415940=43, srv691836265=83, srv1424658731=19, srv1504029417=22, srv1447538654=20, srv1919705034=45, srv212420269=55, srv840346632=93, srv1591220727=27, srv2011898781=48, srv940339403=98, srv223194128=57, srv778955060=88, srv1232484357=12, srv1154047462=9, srv852644063=95, srv1155433441=10, srv386524453=63, srv1141771707=5, srv1952630660=46, srv1797084618=36, srv1596250195=28} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-30 04:53:35,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-30 04:53:35,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-30 04:53:35,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-30 04:53:35,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-30 04:53:35,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-30 04:53:35,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-30 04:53:35,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-30 04:53:35,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-30 04:53:35,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-30 04:53:35,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-30 04:53:35,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-30 04:53:35,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-30 04:53:35,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-30 04:53:35,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-30 04:53:35,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-30 04:53:35,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-30 04:53:35,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-30 04:53:35,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-30 04:53:35,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-30 04:53:35,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-30 04:53:35,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-30 04:53:35,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-30 04:53:35,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-30 04:53:35,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-30 04:53:35,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-30 04:53:35,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-30 04:53:35,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-30 04:53:35,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-30 04:53:35,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-30 04:53:35,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-30 04:53:35,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-30 04:53:35,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-30 04:53:35,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-30 04:53:35,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-30 04:53:35,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-30 04:53:35,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-30 04:53:35,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-30 04:53:35,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-30 04:53:35,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-30 04:53:35,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-30 04:53:35,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-30 04:53:35,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-30 04:53:35,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-30 04:53:35,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-30 04:53:35,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-30 04:53:35,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-30 04:53:35,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-30 04:53:35,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-30 04:53:35,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-30 04:53:35,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-30 04:53:35,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-30 04:53:35,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-30 04:53:35,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-30 04:53:35,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-30 04:53:35,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-30 04:53:35,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-30 04:53:35,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-30 04:53:35,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-30 04:53:35,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-30 04:53:35,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-30 04:53:35,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-30 04:53:35,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-30 04:53:35,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-30 04:53:35,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-30 04:53:35,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-30 04:53:35,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-30 04:53:35,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-30 04:53:35,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-30 04:53:35,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-30 04:53:35,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-30 04:53:35,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-30 04:53:35,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-30 04:53:35,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-30 04:53:35,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-30 04:53:35,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-30 04:53:35,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-30 04:53:35,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-30 04:53:35,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-30 04:53:35,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-30 04:53:35,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-30 04:53:35,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-30 04:53:35,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-30 04:53:35,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-30 04:53:35,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-30 04:53:35,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-30 04:53:35,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-30 04:53:35,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-30 04:53:35,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-30 04:53:35,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-30 04:53:35,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-30 04:53:35,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-30 04:53:35,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-30 04:53:35,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-30 04:53:35,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-30 04:53:35,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-30 04:53:35,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-30 04:53:35,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-30 04:53:35,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-30 04:53:35,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-30 04:53:35,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-30 04:53:35,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-30 04:53:35,750 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=9.537553921509828E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002118805753879094); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-30 04:53:35,750 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1123713316:83 , srv1665697286:83 , srv1726182669:83 , srv212420269:83 , srv223689801:83 , srv42483158:83 , srv496802350:83 , srv538905501:83 , srv1051457520:84 , srv1080346090:84 , srv1094275652:84 , srv1141771707:84 , srv1145695496:84 , srv1145707623:84 , srv1154047462:84 , srv1155433441:84 , srv1213681385:84 , srv1232484357:84 , srv1273903727:84 , srv1299801610:84 , srv1364842120:84 , srv1375163110:84 , srv1383940672:84 , srv1424658731:84 , srv1447538654:84 , srv1450596636:84 , srv1504029417:84 , srv1508232990:84 , srv1529867806:84 , srv1562651081:84 , srv1573494540:84 , srv1591220727:84 , srv1601485067:84 , srv1617216764:84 , srv1689189255:84 , srv1750673279:84 , srv1754415008:84 , srv1797084618:84 , srv1826637472:84 , srv1826989720:84 , srv1836853232:84 , srv1862274765:84 , srv1863107064:84 , srv1866245429:84 , srv1871415940:84 , srv1919705034:84 , srv1952630660:84 , srv1998654150:84 , srv2011898781:84 , srv2017744360:84 , srv2022855665:84 , srv2031402093:84 , srv2051452840:84 , srv210146527:84 , srv2110312128:84 , srv223194128:84 , srv229131794:84 , srv281454017:84 , srv289295242:84 , srv386524453:84 , srv397170164:84 , srv42899276:84 , srv430243969:84 , srv449906673:84 , srv527020219:84 , srv528787434:84 , srv539629846:84 , srv54488484:84 , srv546531135:84 , srv548630484:84 , srv55450174:84 , srv563421758:84 , srv600086848:84 , srv616886713:84 , srv622216481:84 , srv691836265:84 , srv694137785:84 , srv705700741:84 , srv74630509:84 , srv751782089:84 , srv799557335:84 , srv81083320:84 , srv820409871:84 , srv835740405:84 , srv840346632:84 , srv849667549:84 , srv852644063:84 , srv85907115:84 , srv867559295:84 , srv940339403:84 , srv941432090:84 , srv1139774916:85 , srv1267740478:85 , srv1596250195:85 , srv1887091254:85 , srv2143721424:85 , srv257020511:85 , srv528305636:85 , srv778955060:85 , srv1149744427:684 }
2023-04-30 04:53:35,779 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=257 (was 257), ProcessCount=170 (was 170), AvailableMemoryMB=10430 (was 11300)
2023-04-30 04:53:35,790 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=257, ProcessCount=170, AvailableMemoryMB=10430
2023-04-30 04:53:35,791 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-30 04:53:35,791 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-04-30 04:53:35,804 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1035015124:84 , srv1192982049:84 , srv1296658542:84 , srv1541117408:84 , srv1599169009:84 , srv1707679268:84 , srv172314290:84 , srv174937705:84 , srv180177816:84 , srv181030066:84 , srv1965356562:84 , srv2050935845:84 , srv2071837727:84 , srv2135499275:84 , srv232532118:84 , srv406861017:84 , srv447347850:84 , srv466875082:84 , srv499345366:84 , srv52504552:84 , srv528320652:84 , srv533824937:84 , srv556712434:84 , srv702416451:84 , srv710726317:84 , srv736429265:84 , srv741234348:84 , srv88055022:84 , srv90229005:84 , srv1826537222:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-04-30 04:53:35,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-30 04:53:35,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv556712434=23, srv1296658542=2, srv528320652=21, srv1192982049=1, srv2135499275=14, srv2050935845=12, srv1541117408=3, srv710726317=25, srv88055022=28, srv702416451=24, srv466875082=18, srv1826537222=10, srv2071837727=13, srv1965356562=11, srv1707679268=5, srv741234348=27, srv90229005=29, srv174937705=7, srv1035015124=0, srv172314290=6, srv447347850=17, srv406861017=16, srv533824937=22, srv181030066=9, srv736429265=26, srv1599169009=4, srv52504552=20, srv499345366=19, srv232532118=15, srv180177816=8} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-30 04:53:35,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-30 04:53:35,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-30 04:53:35,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-30 04:53:35,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-30 04:53:35,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-30 04:53:35,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-30 04:53:35,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-30 04:53:35,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-30 04:53:35,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-30 04:53:35,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-30 04:53:35,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-30 04:53:35,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-30 04:53:35,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-30 04:53:35,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-30 04:53:35,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-30 04:53:35,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-30 04:53:35,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-30 04:53:35,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-30 04:53:35,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-30 04:53:35,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-30 04:53:35,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-30 04:53:35,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-30 04:53:35,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-30 04:53:35,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-30 04:53:35,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-30 04:53:35,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-30 04:53:35,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-30 04:53:35,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-30 04:53:35,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-30 04:53:35,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-30 04:53:35,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-04-30 04:53:35,836 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0323732003574553, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-30 04:53:35,836 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912295050161526, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0323732003574553, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-04-30 04:55:35,825 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6141905 different iterations.  Found a solution that moves 1850 regions; Going from a computed imbalance of 0.9912295050161526 to a new imbalance of 4.317992290299789E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6851851851851852, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-30 04:55:35,831 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1035015124:90 , srv1192982049:90 , srv1296658542:90 , srv1541117408:90 , srv1599169009:90 , srv1707679268:90 , srv172314290:90 , srv174937705:90 , srv180177816:90 , srv181030066:90 , srv1826537222:90 , srv1965356562:90 , srv2050935845:90 , srv2071837727:90 , srv2135499275:90 , srv232532118:90 , srv406861017:90 , srv447347850:90 , srv466875082:90 , srv499345366:90 , srv52504552:90 , srv528320652:90 , srv533824937:90 , srv556712434:90 , srv702416451:90 , srv710726317:90 , srv736429265:90 , srv741234348:90 , srv88055022:90 , srv90229005:90 }
2023-04-30 04:55:35,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-30 04:55:35,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv556712434=23, srv1296658542=2, srv528320652=21, srv1192982049=1, srv2135499275=14, srv2050935845=12, srv1541117408=3, srv710726317=25, srv88055022=28, srv702416451=24, srv466875082=18, srv1826537222=10, srv2071837727=13, srv1965356562=11, srv1707679268=5, srv741234348=27, srv90229005=29, srv174937705=7, srv1035015124=0, srv172314290=6, srv447347850=17, srv406861017=16, srv533824937=22, srv181030066=9, srv736429265=26, srv1599169009=4, srv52504552=20, srv499345366=19, srv232532118=15, srv180177816=8} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-30 04:55:35,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-30 04:55:35,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-30 04:55:35,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-30 04:55:35,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-30 04:55:35,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-30 04:55:35,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-30 04:55:35,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-30 04:55:35,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-30 04:55:35,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-30 04:55:35,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-30 04:55:35,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-30 04:55:35,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-30 04:55:35,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-30 04:55:35,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-30 04:55:35,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-30 04:55:35,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-30 04:55:35,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-30 04:55:35,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-30 04:55:35,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-30 04:55:35,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-30 04:55:35,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-30 04:55:35,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-30 04:55:35,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-30 04:55:35,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-30 04:55:35,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-30 04:55:35,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-30 04:55:35,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-30 04:55:35,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-30 04:55:35,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-30 04:55:35,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-30 04:55:35,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-04-30 04:55:35,852 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-30 04:55:35,852 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1035015124:90 , srv1192982049:90 , srv1296658542:90 , srv1541117408:90 , srv1599169009:90 , srv1707679268:90 , srv172314290:90 , srv174937705:90 , srv180177816:90 , srv181030066:90 , srv1826537222:90 , srv1965356562:90 , srv2050935845:90 , srv2071837727:90 , srv2135499275:90 , srv232532118:90 , srv406861017:90 , srv447347850:90 , srv466875082:90 , srv499345366:90 , srv52504552:90 , srv528320652:90 , srv533824937:90 , srv556712434:90 , srv702416451:90 , srv710726317:90 , srv736429265:90 , srv741234348:90 , srv88055022:90 , srv90229005:90 }
2023-04-30 04:55:35,870 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=183 (was 257), ProcessCount=170 (was 170), AvailableMemoryMB=10649 (was 10430) - AvailableMemoryMB LEAK? -