2024-01-15 09:53:19,199 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2024-01-15 09:53:19,531 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2024-01-15 09:53:19,649 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2024-01-15 09:53:19,723 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-01-15 09:53:19,731 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-01-15 09:53:19,744 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=286, ProcessCount=167, AvailableMemoryMB=15130
2024-01-15 09:53:19,746 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-01-15 09:53:19,747 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-01-15 09:53:19,880 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1002455356:84 , srv103782892:84 , srv1076039131:84 , srv108395837:84 , srv1090023692:84 , srv111758012:84 , srv1126918460:84 , srv1128581694:84 , srv1215966899:84 , srv1220855004:84 , srv1234820265:84 , srv1234859375:84 , srv1244076644:84 , srv1268162622:84 , srv1275501646:84 , srv1275654051:84 , srv1337337400:84 , srv134301949:84 , srv1347262222:84 , srv1354385606:84 , srv1382371943:84 , srv1394624167:84 , srv1453023226:84 , srv1455536078:84 , srv1464355497:84 , srv1510170785:84 , srv151321537:84 , srv1518215144:84 , srv1526447518:84 , srv1531936854:84 , srv1544156623:84 , srv156809153:84 , srv1572174777:84 , srv1581413565:84 , srv1592835302:84 , srv1603522143:84 , srv1607122872:84 , srv165027652:84 , srv1673435253:84 , srv1682819961:84 , srv1683263250:84 , srv169956856:84 , srv1777309069:84 , srv1777398346:84 , srv1778503244:84 , srv1795339152:84 , srv1816561880:84 , srv1881541677:84 , srv1924103799:84 , srv1943920772:84 , srv1944362160:84 , srv1956089663:84 , srv1969616689:84 , srv1989411193:84 , srv1989736922:84 , srv2046372550:84 , srv2054646227:84 , srv2067476677:84 , srv2080426682:84 , srv2084218864:84 , srv2085272482:84 , srv21041165:84 , srv2118894492:84 , srv2122762435:84 , srv214182253:84 , srv243197673:84 , srv281001750:84 , srv293098761:84 , srv294728847:84 , srv321369784:84 , srv322416354:84 , srv323829143:84 , srv369242239:84 , srv439937700:84 , srv453727220:84 , srv461830429:84 , srv498368520:84 , srv49909331:84 , srv551346764:84 , srv585791256:84 , srv632405176:84 , srv639994593:84 , srv646201600:84 , srv672703913:84 , srv718634315:84 , srv718637484:84 , srv796542873:84 , srv821392048:84 , srv821648689:84 , srv839140648:84 , srv867475293:84 , srv877097957:84 , srv877280984:84 , srv890948332:84 , srv910435323:84 , srv922184644:84 , srv944980225:84 , srv960282199:84 , srv964713103:84 , srv855202663:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2024-01-15 09:53:19,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-15 09:53:19,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1603522143=35, srv214182253=64, srv639994593=81, srv646201600=82, srv821392048=87, srv632405176=80, srv1275654051=15, srv1572174777=32, srv585791256=79, srv1394624167=21, srv1956089663=51, srv1777398346=43, srv498368520=76, srv890948332=94, srv243197673=65, srv1518215144=27, srv2080426682=58, srv461830429=75, srv1943920772=49, srv151321537=26, srv293098761=67, srv1275501646=14, srv21041165=61, srv1128581694=7, srv1510170785=25, srv169956856=41, srv281001750=66, srv1581413565=33, srv323829143=71, srv1455536078=23, srv1989736922=54, srv439937700=73, srv1126918460=6, srv1464355497=24, srv1881541677=47, srv1682819961=39, srv2084218864=59, srv1989411193=53, srv1607122872=36, srv1592835302=34, srv2118894492=62, srv718637484=85, srv1795339152=45, srv796542873=86, srv2067476677=57, srv453727220=74, srv1526447518=28, srv1531936854=29, srv1924103799=48, srv2122762435=63, srv551346764=78, srv718634315=84, srv111758012=5, srv1215966899=8, srv839140648=89, srv1268162622=13, srv2054646227=56, srv960282199=98, srv103782892=1, srv944980225=97, srv1002455356=0, srv294728847=68, srv369242239=72, srv1354385606=19, srv1382371943=20, srv49909331=77, srv1778503244=44, srv156809153=31, srv1944362160=50, srv1544156623=30, srv108395837=3, srv1090023692=4, srv134301949=17, srv1969616689=52, srv964713103=99, srv877280984=93, srv672703913=83, srv1244076644=12, srv1347262222=18, srv1453023226=22, srv855202663=90, srv1816561880=46, srv321369784=69, srv165027652=37, srv867475293=91, srv2085272482=60, srv1234820265=10, srv1076039131=2, srv1234859375=11, srv322416354=70, srv1337337400=16, srv1673435253=38, srv922184644=96, srv1220855004=9, srv1777309069=42, srv1683263250=40, srv877097957=92, srv2046372550=55, srv910435323=95, srv821648689=88} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-15 09:53:19,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-15 09:53:19,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-15 09:53:19,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-15 09:53:19,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-15 09:53:19,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-15 09:53:19,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-15 09:53:19,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-15 09:53:19,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-15 09:53:19,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-15 09:53:19,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-15 09:53:19,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-15 09:53:19,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-15 09:53:19,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-15 09:53:19,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-15 09:53:19,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-15 09:53:19,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-15 09:53:19,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-15 09:53:19,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-15 09:53:19,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-15 09:53:19,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-15 09:53:19,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-15 09:53:19,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-15 09:53:19,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-15 09:53:19,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-15 09:53:19,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-15 09:53:19,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-15 09:53:19,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-15 09:53:19,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-15 09:53:19,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-15 09:53:19,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-15 09:53:19,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-01-15 09:53:19,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-01-15 09:53:19,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-01-15 09:53:19,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-01-15 09:53:19,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-01-15 09:53:19,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-01-15 09:53:19,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-01-15 09:53:19,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-01-15 09:53:19,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-01-15 09:53:19,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-01-15 09:53:19,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-01-15 09:53:19,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-01-15 09:53:19,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-01-15 09:53:19,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-01-15 09:53:19,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-01-15 09:53:19,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-01-15 09:53:19,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-01-15 09:53:19,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-01-15 09:53:19,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-01-15 09:53:19,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-01-15 09:53:19,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-01-15 09:53:19,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-01-15 09:53:19,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-01-15 09:53:19,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-01-15 09:53:19,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-01-15 09:53:19,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-01-15 09:53:19,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-01-15 09:53:19,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-01-15 09:53:19,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-01-15 09:53:19,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-01-15 09:53:19,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-01-15 09:53:19,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-01-15 09:53:19,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-01-15 09:53:19,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-01-15 09:53:19,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-01-15 09:53:19,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-01-15 09:53:19,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-01-15 09:53:19,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-01-15 09:53:19,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-01-15 09:53:19,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-01-15 09:53:19,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-01-15 09:53:19,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-01-15 09:53:19,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-01-15 09:53:19,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-01-15 09:53:19,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-01-15 09:53:19,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-01-15 09:53:19,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-01-15 09:53:19,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-01-15 09:53:19,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-01-15 09:53:19,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-01-15 09:53:19,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-01-15 09:53:19,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-01-15 09:53:19,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-01-15 09:53:19,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-01-15 09:53:19,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-01-15 09:53:19,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-01-15 09:53:19,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-01-15 09:53:19,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-01-15 09:53:19,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-01-15 09:53:19,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-01-15 09:53:19,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-01-15 09:53:19,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-01-15 09:53:19,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-01-15 09:53:19,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-01-15 09:53:19,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-01-15 09:53:19,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-01-15 09:53:19,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-01-15 09:53:19,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-01-15 09:53:19,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-01-15 09:53:19,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-01-15 09:53:19,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-01-15 09:53:20,212 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-15 09:53:20,212 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-01-15 09:53:20,213 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-01-15 09:53:20,642 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 625 ms to try 5000 different iterations.  Found a solution that moves 4681 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1803404765059814. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0405243718803733, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05771558424223043, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5201111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0405243718803733, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17558333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24191666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-15 09:53:20,660 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1795339152:81 , srv1464355497:82 , srv1518215144:82 , srv821392048:82 , srv1244076644:83 , srv1275501646:83 , srv1607122872:83 , srv1778503244:83 , srv439937700:83 , srv718637484:83 , srv1234859375:84 , srv1275654051:84 , srv1347262222:84 , srv156809153:84 , srv1572174777:84 , srv1581413565:84 , srv1603522143:84 , srv1881541677:84 , srv2054646227:84 , srv2085272482:84 , srv718634315:84 , srv910435323:84 , srv108395837:85 , srv1215966899:85 , srv1337337400:85 , srv1354385606:85 , srv1453023226:85 , srv1526447518:85 , srv1544156623:85 , srv1592835302:85 , srv1682819961:85 , srv1683263250:85 , srv1969616689:85 , srv1989411193:85 , srv2067476677:85 , srv294728847:85 , srv323829143:85 , srv461830429:85 , srv49909331:85 , srv877097957:85 , srv877280984:85 , srv964713103:85 , srv1090023692:86 , srv111758012:86 , srv134301949:86 , srv1455536078:86 , srv151321537:86 , srv1956089663:86 , srv1989736922:86 , srv2084218864:86 , srv21041165:86 , srv243197673:86 , srv321369784:86 , srv585791256:86 , srv632405176:86 , srv796542873:86 , srv821648689:86 , srv867475293:86 , srv922184644:86 , srv944980225:86 , srv1220855004:87 , srv1234820265:87 , srv1268162622:87 , srv1510170785:87 , srv1944362160:87 , srv2080426682:87 , srv2122762435:87 , srv281001750:87 , srv498368520:87 , srv839140648:87 , srv960282199:87 , srv1076039131:88 , srv1531936854:88 , srv169956856:88 , srv293098761:88 , srv369242239:88 , srv639994593:88 , srv646201600:88 , srv890948332:88 , srv103782892:89 , srv1394624167:89 , srv165027652:89 , srv1673435253:89 , srv1816561880:89 , srv1924103799:89 , srv1943920772:89 , srv2046372550:89 , srv551346764:89 , srv672703913:89 , srv1002455356:90 , srv1128581694:90 , srv1382371943:90 , srv1777398346:90 , srv2118894492:90 , srv214182253:90 , srv453727220:90 , srv1126918460:93 , srv322416354:93 , srv1777309069:102 , srv855202663:450 }
2024-01-15 09:53:20,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-15 09:53:20,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1603522143=35, srv214182253=64, srv639994593=81, srv646201600=82, srv821392048=87, srv632405176=80, srv1275654051=15, srv1572174777=32, srv585791256=79, srv1394624167=21, srv1956089663=51, srv1777398346=43, srv498368520=76, srv890948332=94, srv243197673=65, srv1518215144=27, srv2080426682=58, srv461830429=75, srv1943920772=49, srv151321537=26, srv293098761=67, srv1275501646=14, srv21041165=61, srv1128581694=7, srv1510170785=25, srv169956856=41, srv281001750=66, srv1581413565=33, srv323829143=71, srv1455536078=23, srv1989736922=54, srv439937700=73, srv1126918460=6, srv1464355497=24, srv1881541677=47, srv1682819961=39, srv2084218864=59, srv1989411193=53, srv1607122872=36, srv1592835302=34, srv2118894492=62, srv718637484=85, srv1795339152=45, srv796542873=86, srv2067476677=57, srv453727220=74, srv1526447518=28, srv1531936854=29, srv1924103799=48, srv2122762435=63, srv551346764=78, srv718634315=84, srv111758012=5, srv1215966899=8, srv839140648=89, srv1268162622=13, srv2054646227=56, srv960282199=98, srv103782892=1, srv944980225=97, srv1002455356=0, srv294728847=68, srv369242239=72, srv1354385606=19, srv1382371943=20, srv49909331=77, srv1778503244=44, srv156809153=31, srv1944362160=50, srv1544156623=30, srv108395837=3, srv1090023692=4, srv134301949=17, srv1969616689=52, srv964713103=99, srv877280984=93, srv672703913=83, srv1244076644=12, srv1347262222=18, srv1453023226=22, srv855202663=90, srv1816561880=46, srv321369784=69, srv165027652=37, srv867475293=91, srv2085272482=60, srv1234820265=10, srv1076039131=2, srv1234859375=11, srv322416354=70, srv1337337400=16, srv1673435253=38, srv922184644=96, srv1220855004=9, srv1777309069=42, srv1683263250=40, srv877097957=92, srv2046372550=55, srv910435323=95, srv821648689=88} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-15 09:53:20,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-15 09:53:20,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-15 09:53:20,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-15 09:53:20,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-15 09:53:20,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-15 09:53:20,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-15 09:53:20,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-15 09:53:20,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-15 09:53:20,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-15 09:53:20,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-15 09:53:20,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-15 09:53:20,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-15 09:53:20,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-15 09:53:20,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-15 09:53:20,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-15 09:53:20,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-15 09:53:20,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-15 09:53:20,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-15 09:53:20,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-15 09:53:20,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-15 09:53:20,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-15 09:53:20,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-15 09:53:20,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-15 09:53:20,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-15 09:53:20,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-15 09:53:20,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-15 09:53:20,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-15 09:53:20,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-15 09:53:20,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-15 09:53:20,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-15 09:53:20,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-01-15 09:53:20,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-01-15 09:53:20,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-01-15 09:53:20,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-01-15 09:53:20,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-01-15 09:53:20,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-01-15 09:53:20,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-01-15 09:53:20,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-01-15 09:53:20,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-01-15 09:53:20,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-01-15 09:53:20,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-01-15 09:53:20,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-01-15 09:53:20,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-01-15 09:53:20,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-01-15 09:53:20,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-01-15 09:53:20,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-01-15 09:53:20,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-01-15 09:53:20,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-01-15 09:53:20,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-01-15 09:53:20,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-01-15 09:53:20,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-01-15 09:53:20,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-01-15 09:53:20,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-01-15 09:53:20,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-01-15 09:53:20,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-01-15 09:53:20,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-01-15 09:53:20,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-01-15 09:53:20,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-01-15 09:53:20,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-01-15 09:53:20,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-01-15 09:53:20,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-01-15 09:53:20,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-01-15 09:53:20,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-01-15 09:53:20,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-01-15 09:53:20,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-01-15 09:53:20,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-01-15 09:53:20,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-01-15 09:53:20,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-01-15 09:53:20,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-01-15 09:53:20,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-01-15 09:53:20,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-01-15 09:53:20,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-01-15 09:53:20,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-01-15 09:53:20,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-01-15 09:53:20,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-01-15 09:53:20,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-01-15 09:53:20,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-01-15 09:53:20,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-01-15 09:53:20,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-01-15 09:53:20,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-01-15 09:53:20,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-01-15 09:53:20,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-01-15 09:53:20,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-01-15 09:53:20,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-01-15 09:53:20,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-01-15 09:53:20,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-01-15 09:53:20,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-01-15 09:53:20,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-01-15 09:53:20,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-01-15 09:53:20,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-01-15 09:53:20,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-01-15 09:53:20,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-01-15 09:53:20,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-01-15 09:53:20,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-01-15 09:53:20,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-01-15 09:53:20,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-01-15 09:53:20,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-01-15 09:53:20,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-01-15 09:53:20,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-01-15 09:53:20,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-01-15 09:53:20,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-01-15 09:53:20,828 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0405243718803733, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05771558424223043, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0405243718803733, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17558333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24191666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-15 09:53:20,828 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-01-15 09:53:20,829 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.1803076994434232, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0405243718803733, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05771558424223043, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0405243718803733, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17558333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24191666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-01-15 09:53:21,115 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 407 ms to try 5000 different iterations.  Found a solution that moves 2643 regions; Going from a computed imbalance of 0.1803076994434232 to a new imbalance of 0.015288258542478103. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02206623140457042); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.030292253976123732, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2936666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02206623140457042); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012416666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04275, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-15 09:53:21,132 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1777309069:69 , srv1126918460:78 , srv322416354:78 , srv453727220:79 , srv103782892:81 , srv165027652:81 , srv1777398346:81 , srv1816561880:81 , srv1531936854:82 , srv1924103799:82 , srv2118894492:82 , srv214182253:82 , srv1002455356:83 , srv1128581694:83 , srv1234820265:83 , srv1394624167:83 , srv1673435253:83 , srv281001750:83 , srv369242239:83 , srv672703913:83 , srv839140648:83 , srv960282199:83 , srv1453023226:84 , srv1526447518:84 , srv169956856:84 , srv2046372550:84 , srv2084218864:84 , srv551346764:84 , srv821648689:84 , srv890948332:84 , srv944980225:84 , srv1382371943:85 , srv151321537:85 , srv1943920772:85 , srv498368520:85 , srv646201600:85 , srv796542873:85 , srv877280984:85 , srv1090023692:86 , srv1268162622:86 , srv1347262222:86 , srv1354385606:86 , srv1455536078:86 , srv1510170785:86 , srv156809153:86 , srv1572174777:86 , srv1944362160:86 , srv2080426682:86 , srv2085272482:86 , srv243197673:86 , srv293098761:86 , srv321369784:86 , srv461830429:86 , srv910435323:86 , srv964713103:86 , srv108395837:87 , srv111758012:87 , srv1215966899:87 , srv1220855004:87 , srv1234859375:87 , srv134301949:87 , srv1544156623:87 , srv1581413565:87 , srv1956089663:87 , srv1969616689:87 , srv1989736922:87 , srv2122762435:87 , srv439937700:87 , srv585791256:87 , srv639994593:87 , srv867475293:87 , srv1275501646:88 , srv1275654051:88 , srv1337337400:88 , srv1603522143:88 , srv1881541677:88 , srv1989411193:88 , srv2067476677:88 , srv21041165:88 , srv294728847:88 , srv49909331:88 , srv922184644:88 , srv1076039131:89 , srv1518215144:89 , srv1592835302:89 , srv1607122872:89 , srv1682819961:89 , srv1683263250:89 , srv1778503244:89 , srv2054646227:89 , srv632405176:89 , srv718634315:89 , srv877097957:89 , srv1464355497:90 , srv323829143:90 , srv718637484:90 , srv821392048:90 , srv1244076644:91 , srv1795339152:91 , srv855202663:520 }
2024-01-15 09:53:21,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-15 09:53:21,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1603522143=35, srv214182253=64, srv639994593=81, srv646201600=82, srv821392048=87, srv632405176=80, srv1275654051=15, srv1572174777=32, srv585791256=79, srv1394624167=21, srv1956089663=51, srv1777398346=43, srv498368520=76, srv890948332=94, srv243197673=65, srv1518215144=27, srv2080426682=58, srv461830429=75, srv1943920772=49, srv151321537=26, srv293098761=67, srv1275501646=14, srv21041165=61, srv1128581694=7, srv1510170785=25, srv169956856=41, srv281001750=66, srv1581413565=33, srv323829143=71, srv1455536078=23, srv1989736922=54, srv439937700=73, srv1126918460=6, srv1464355497=24, srv1881541677=47, srv1682819961=39, srv2084218864=59, srv1989411193=53, srv1607122872=36, srv1592835302=34, srv2118894492=62, srv718637484=85, srv1795339152=45, srv796542873=86, srv2067476677=57, srv453727220=74, srv1526447518=28, srv1531936854=29, srv1924103799=48, srv2122762435=63, srv551346764=78, srv718634315=84, srv111758012=5, srv1215966899=8, srv839140648=89, srv1268162622=13, srv2054646227=56, srv960282199=98, srv103782892=1, srv944980225=97, srv1002455356=0, srv294728847=68, srv369242239=72, srv1354385606=19, srv1382371943=20, srv49909331=77, srv1778503244=44, srv156809153=31, srv1944362160=50, srv1544156623=30, srv108395837=3, srv1090023692=4, srv134301949=17, srv1969616689=52, srv964713103=99, srv877280984=93, srv672703913=83, srv1244076644=12, srv1347262222=18, srv1453023226=22, srv855202663=90, srv1816561880=46, srv321369784=69, srv165027652=37, srv867475293=91, srv2085272482=60, srv1234820265=10, srv1076039131=2, srv1234859375=11, srv322416354=70, srv1337337400=16, srv1673435253=38, srv922184644=96, srv1220855004=9, srv1777309069=42, srv1683263250=40, srv877097957=92, srv2046372550=55, srv910435323=95, srv821648689=88} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-15 09:53:21,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-15 09:53:21,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-15 09:53:21,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-15 09:53:21,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-15 09:53:21,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-15 09:53:21,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-15 09:53:21,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-15 09:53:21,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-15 09:53:21,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-15 09:53:21,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-15 09:53:21,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-15 09:53:21,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-15 09:53:21,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-15 09:53:21,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-15 09:53:21,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-15 09:53:21,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-15 09:53:21,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-15 09:53:21,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-15 09:53:21,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-15 09:53:21,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-15 09:53:21,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-15 09:53:21,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-15 09:53:21,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-15 09:53:21,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-15 09:53:21,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-15 09:53:21,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-15 09:53:21,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-15 09:53:21,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-15 09:53:21,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-15 09:53:21,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-15 09:53:21,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-01-15 09:53:21,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-01-15 09:53:21,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-01-15 09:53:21,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-01-15 09:53:21,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-01-15 09:53:21,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-01-15 09:53:21,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-01-15 09:53:21,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-01-15 09:53:21,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-01-15 09:53:21,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-01-15 09:53:21,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-01-15 09:53:21,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-01-15 09:53:21,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-01-15 09:53:21,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-01-15 09:53:21,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-01-15 09:53:21,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-01-15 09:53:21,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-01-15 09:53:21,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-01-15 09:53:21,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-01-15 09:53:21,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-01-15 09:53:21,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-01-15 09:53:21,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-01-15 09:53:21,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-01-15 09:53:21,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-01-15 09:53:21,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-01-15 09:53:21,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-01-15 09:53:21,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-01-15 09:53:21,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-01-15 09:53:21,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-01-15 09:53:21,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-01-15 09:53:21,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-01-15 09:53:21,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-01-15 09:53:21,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-01-15 09:53:21,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-01-15 09:53:21,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-01-15 09:53:21,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-01-15 09:53:21,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-01-15 09:53:21,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-01-15 09:53:21,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-01-15 09:53:21,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-01-15 09:53:21,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-01-15 09:53:21,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-01-15 09:53:21,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-01-15 09:53:21,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-01-15 09:53:21,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-01-15 09:53:21,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-01-15 09:53:21,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-01-15 09:53:21,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-01-15 09:53:21,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-01-15 09:53:21,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-01-15 09:53:21,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-01-15 09:53:21,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-01-15 09:53:21,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-01-15 09:53:21,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-01-15 09:53:21,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-01-15 09:53:21,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-01-15 09:53:21,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-01-15 09:53:21,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-01-15 09:53:21,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-01-15 09:53:21,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-01-15 09:53:21,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-01-15 09:53:21,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-01-15 09:53:21,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-01-15 09:53:21,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-01-15 09:53:21,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-01-15 09:53:21,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-01-15 09:53:21,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-01-15 09:53:21,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-01-15 09:53:21,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-01-15 09:53:21,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-01-15 09:53:21,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-01-15 09:53:21,289 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02206623140457042); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.030292253976123732, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02206623140457042); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012416666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04275, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-15 09:53:21,289 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-01-15 09:53:21,289 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.015269751860926868, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02206623140457042); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.030292253976123732, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02206623140457042); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012416666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04275, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-01-15 09:53:21,538 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 349 ms to try 5000 different iterations.  Found a solution that moves 937 regions; Going from a computed imbalance of 0.015269751860926868 to a new imbalance of 6.461705544319983E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0062135581815069485); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005443310539518174); MoveCostFunction : (multiplier=7.0, imbalance=0.10411111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0062135581815069485); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-15 09:53:21,540 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1076039131:81 , srv1382371943:82 , srv1683263250:82 , srv1943920772:82 , srv639994593:82 , srv1220855004:83 , srv1592835302:83 , srv1682819961:83 , srv1989411193:83 , srv21041165:83 , srv632405176:83 , srv922184644:83 , srv1002455356:84 , srv111758012:84 , srv1128581694:84 , srv1244076644:84 , srv1268162622:84 , srv1337337400:84 , srv1510170785:84 , srv1795339152:84 , srv1956089663:84 , srv2054646227:84 , srv2067476677:84 , srv2122762435:84 , srv293098761:84 , srv294728847:84 , srv323829143:84 , srv551346764:84 , srv646201600:84 , srv877097957:84 , srv134301949:85 , srv1394624167:85 , srv1464355497:85 , srv1544156623:85 , srv169956856:85 , srv1778503244:85 , srv1881541677:85 , srv1944362160:85 , srv1989736922:85 , srv2046372550:85 , srv2080426682:85 , srv2118894492:85 , srv214182253:85 , srv243197673:85 , srv321369784:85 , srv498368520:85 , srv585791256:85 , srv718634315:85 , srv718637484:85 , srv821392048:85 , srv867475293:85 , srv890948332:85 , srv108395837:86 , srv1090023692:86 , srv1126918460:86 , srv1215966899:86 , srv1275501646:86 , srv1275654051:86 , srv1354385606:86 , srv1455536078:86 , srv1603522143:86 , srv1607122872:86 , srv1673435253:86 , srv1969616689:86 , srv322416354:86 , srv369242239:86 , srv461830429:86 , srv49909331:86 , srv672703913:86 , srv796542873:86 , srv103782892:87 , srv1234820265:87 , srv1234859375:87 , srv1347262222:87 , srv151321537:87 , srv1518215144:87 , srv1531936854:87 , srv156809153:87 , srv1572174777:87 , srv1581413565:87 , srv165027652:87 , srv1777309069:87 , srv1816561880:87 , srv1924103799:87 , srv439937700:87 , srv821648689:87 , srv944980225:87 , srv964713103:87 , srv1526447518:88 , srv1777398346:88 , srv2084218864:88 , srv2085272482:88 , srv839140648:88 , srv877280984:88 , srv910435323:88 , srv960282199:88 , srv1453023226:89 , srv281001750:89 , srv453727220:89 , srv855202663:543 }
2024-01-15 09:53:21,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-15 09:53:21,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1603522143=35, srv214182253=64, srv639994593=81, srv646201600=82, srv821392048=87, srv632405176=80, srv1275654051=15, srv1572174777=32, srv585791256=79, srv1394624167=21, srv1956089663=51, srv1777398346=43, srv498368520=76, srv890948332=94, srv243197673=65, srv1518215144=27, srv2080426682=58, srv461830429=75, srv1943920772=49, srv151321537=26, srv293098761=67, srv1275501646=14, srv21041165=61, srv1128581694=7, srv1510170785=25, srv169956856=41, srv281001750=66, srv1581413565=33, srv323829143=71, srv1455536078=23, srv1989736922=54, srv439937700=73, srv1126918460=6, srv1464355497=24, srv1881541677=47, srv1682819961=39, srv2084218864=59, srv1989411193=53, srv1607122872=36, srv1592835302=34, srv2118894492=62, srv718637484=85, srv1795339152=45, srv796542873=86, srv2067476677=57, srv453727220=74, srv1526447518=28, srv1531936854=29, srv1924103799=48, srv2122762435=63, srv551346764=78, srv718634315=84, srv111758012=5, srv1215966899=8, srv839140648=89, srv1268162622=13, srv2054646227=56, srv960282199=98, srv103782892=1, srv944980225=97, srv1002455356=0, srv294728847=68, srv369242239=72, srv1354385606=19, srv1382371943=20, srv49909331=77, srv1778503244=44, srv156809153=31, srv1944362160=50, srv1544156623=30, srv108395837=3, srv1090023692=4, srv134301949=17, srv1969616689=52, srv964713103=99, srv877280984=93, srv672703913=83, srv1244076644=12, srv1347262222=18, srv1453023226=22, srv855202663=90, srv1816561880=46, srv321369784=69, srv165027652=37, srv867475293=91, srv2085272482=60, srv1234820265=10, srv1076039131=2, srv1234859375=11, srv322416354=70, srv1337337400=16, srv1673435253=38, srv922184644=96, srv1220855004=9, srv1777309069=42, srv1683263250=40, srv877097957=92, srv2046372550=55, srv910435323=95, srv821648689=88} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-15 09:53:21,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-15 09:53:21,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-15 09:53:21,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-15 09:53:21,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-15 09:53:21,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-15 09:53:21,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-15 09:53:21,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-15 09:53:21,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-15 09:53:21,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-15 09:53:21,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-15 09:53:21,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-15 09:53:21,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-15 09:53:21,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-15 09:53:21,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-15 09:53:21,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-15 09:53:21,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-15 09:53:21,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-15 09:53:21,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-15 09:53:21,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-15 09:53:21,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-15 09:53:21,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-15 09:53:21,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-15 09:53:21,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-15 09:53:21,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-15 09:53:21,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-15 09:53:21,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-15 09:53:21,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-15 09:53:21,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-15 09:53:21,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-15 09:53:21,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-15 09:53:21,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-01-15 09:53:21,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-01-15 09:53:21,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-01-15 09:53:21,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-01-15 09:53:21,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-01-15 09:53:21,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-01-15 09:53:21,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-01-15 09:53:21,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-01-15 09:53:21,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-01-15 09:53:21,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-01-15 09:53:21,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-01-15 09:53:21,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-01-15 09:53:21,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-01-15 09:53:21,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-01-15 09:53:21,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-01-15 09:53:21,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-01-15 09:53:21,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-01-15 09:53:21,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-01-15 09:53:21,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-01-15 09:53:21,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-01-15 09:53:21,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-01-15 09:53:21,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-01-15 09:53:21,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-01-15 09:53:21,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-01-15 09:53:21,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-01-15 09:53:21,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-01-15 09:53:21,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-01-15 09:53:21,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-01-15 09:53:21,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-01-15 09:53:21,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-01-15 09:53:21,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-01-15 09:53:21,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-01-15 09:53:21,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-01-15 09:53:21,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-01-15 09:53:21,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-01-15 09:53:21,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-01-15 09:53:21,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-01-15 09:53:21,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-01-15 09:53:21,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-01-15 09:53:21,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-01-15 09:53:21,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-01-15 09:53:21,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-01-15 09:53:21,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-01-15 09:53:21,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-01-15 09:53:21,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-01-15 09:53:21,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-01-15 09:53:21,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-01-15 09:53:21,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-01-15 09:53:21,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-01-15 09:53:21,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-01-15 09:53:21,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-01-15 09:53:21,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-01-15 09:53:21,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-01-15 09:53:21,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-01-15 09:53:21,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-01-15 09:53:21,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-01-15 09:53:21,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-01-15 09:53:21,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-01-15 09:53:21,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-01-15 09:53:21,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-01-15 09:53:21,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-01-15 09:53:21,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-01-15 09:53:21,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-01-15 09:53:21,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-01-15 09:53:21,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-01-15 09:53:21,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-01-15 09:53:21,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-01-15 09:53:21,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-01-15 09:53:21,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-01-15 09:53:21,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-01-15 09:53:21,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-01-15 09:53:21,690 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0062135581815069485); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005443310539518174); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0062135581815069485); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-15 09:53:21,690 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-01-15 09:53:21,690 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=6.396095402006293E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0062135581815069485); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005443310539518174); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0062135581815069485); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-01-15 09:53:22,056 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 466 ms to try 5000 different iterations.  Found a solution that moves 252 regions; Going from a computed imbalance of 6.396095402006293E-4 to a new imbalance of 4.200526588834011E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.5313429231318235E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.028, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.5313429231318235E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-15 09:53:22,057 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1777398346:83 , srv281001750:83 , srv323829143:83 , srv49909331:83 , srv108395837:84 , srv1090023692:84 , srv1126918460:84 , srv1215966899:84 , srv1234820265:84 , srv1234859375:84 , srv1244076644:84 , srv1275654051:84 , srv134301949:84 , srv1453023226:84 , srv1455536078:84 , srv151321537:84 , srv1518215144:84 , srv156809153:84 , srv1581413565:84 , srv1603522143:84 , srv1607122872:84 , srv1673435253:84 , srv1777309069:84 , srv1816561880:84 , srv1924103799:84 , srv1944362160:84 , srv1969616689:84 , srv1989736922:84 , srv2046372550:84 , srv2080426682:84 , srv2084218864:84 , srv2085272482:84 , srv2122762435:84 , srv293098761:84 , srv453727220:84 , srv632405176:84 , srv672703913:84 , srv718634315:84 , srv718637484:84 , srv821392048:84 , srv839140648:84 , srv867475293:84 , srv877097957:84 , srv877280984:84 , srv910435323:84 , srv960282199:84 , srv964713103:84 , srv1002455356:85 , srv103782892:85 , srv1076039131:85 , srv111758012:85 , srv1128581694:85 , srv1220855004:85 , srv1268162622:85 , srv1275501646:85 , srv1337337400:85 , srv1347262222:85 , srv1354385606:85 , srv1382371943:85 , srv1394624167:85 , srv1510170785:85 , srv1526447518:85 , srv1531936854:85 , srv1544156623:85 , srv1572174777:85 , srv1592835302:85 , srv165027652:85 , srv1682819961:85 , srv169956856:85 , srv1778503244:85 , srv1881541677:85 , srv1956089663:85 , srv2054646227:85 , srv2067476677:85 , srv21041165:85 , srv2118894492:85 , srv214182253:85 , srv294728847:85 , srv321369784:85 , srv322416354:85 , srv369242239:85 , srv439937700:85 , srv461830429:85 , srv498368520:85 , srv551346764:85 , srv585791256:85 , srv646201600:85 , srv796542873:85 , srv821648689:85 , srv890948332:85 , srv922184644:85 , srv944980225:85 , srv1464355497:86 , srv1683263250:86 , srv1795339152:86 , srv1943920772:86 , srv1989411193:86 , srv243197673:86 , srv639994593:86 , srv855202663:629 }
2024-01-15 09:53:22,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-15 09:53:22,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1603522143=35, srv214182253=64, srv639994593=81, srv646201600=82, srv821392048=87, srv632405176=80, srv1275654051=15, srv1572174777=32, srv585791256=79, srv1394624167=21, srv1956089663=51, srv1777398346=43, srv498368520=76, srv890948332=94, srv243197673=65, srv1518215144=27, srv2080426682=58, srv461830429=75, srv1943920772=49, srv151321537=26, srv293098761=67, srv1275501646=14, srv21041165=61, srv1128581694=7, srv1510170785=25, srv169956856=41, srv281001750=66, srv1581413565=33, srv323829143=71, srv1455536078=23, srv1989736922=54, srv439937700=73, srv1126918460=6, srv1464355497=24, srv1881541677=47, srv1682819961=39, srv2084218864=59, srv1989411193=53, srv1607122872=36, srv1592835302=34, srv2118894492=62, srv718637484=85, srv1795339152=45, srv796542873=86, srv2067476677=57, srv453727220=74, srv1526447518=28, srv1531936854=29, srv1924103799=48, srv2122762435=63, srv551346764=78, srv718634315=84, srv111758012=5, srv1215966899=8, srv839140648=89, srv1268162622=13, srv2054646227=56, srv960282199=98, srv103782892=1, srv944980225=97, srv1002455356=0, srv294728847=68, srv369242239=72, srv1354385606=19, srv1382371943=20, srv49909331=77, srv1778503244=44, srv156809153=31, srv1944362160=50, srv1544156623=30, srv108395837=3, srv1090023692=4, srv134301949=17, srv1969616689=52, srv964713103=99, srv877280984=93, srv672703913=83, srv1244076644=12, srv1347262222=18, srv1453023226=22, srv855202663=90, srv1816561880=46, srv321369784=69, srv165027652=37, srv867475293=91, srv2085272482=60, srv1234820265=10, srv1076039131=2, srv1234859375=11, srv322416354=70, srv1337337400=16, srv1673435253=38, srv922184644=96, srv1220855004=9, srv1777309069=42, srv1683263250=40, srv877097957=92, srv2046372550=55, srv910435323=95, srv821648689=88} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-15 09:53:22,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-15 09:53:22,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-15 09:53:22,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-15 09:53:22,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-15 09:53:22,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-15 09:53:22,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-15 09:53:22,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-15 09:53:22,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-15 09:53:22,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-15 09:53:22,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-15 09:53:22,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-15 09:53:22,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-15 09:53:22,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-15 09:53:22,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-15 09:53:22,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-15 09:53:22,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-15 09:53:22,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-15 09:53:22,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-15 09:53:22,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-15 09:53:22,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-15 09:53:22,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-15 09:53:22,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-15 09:53:22,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-15 09:53:22,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-15 09:53:22,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-15 09:53:22,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-15 09:53:22,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-15 09:53:22,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-15 09:53:22,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-15 09:53:22,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-15 09:53:22,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-01-15 09:53:22,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-01-15 09:53:22,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-01-15 09:53:22,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-01-15 09:53:22,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-01-15 09:53:22,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-01-15 09:53:22,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-01-15 09:53:22,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-01-15 09:53:22,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-01-15 09:53:22,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-01-15 09:53:22,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-01-15 09:53:22,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-01-15 09:53:22,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-01-15 09:53:22,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-01-15 09:53:22,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-01-15 09:53:22,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-01-15 09:53:22,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-01-15 09:53:22,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-01-15 09:53:22,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-01-15 09:53:22,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-01-15 09:53:22,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-01-15 09:53:22,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-01-15 09:53:22,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-01-15 09:53:22,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-01-15 09:53:22,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-01-15 09:53:22,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-01-15 09:53:22,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-01-15 09:53:22,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-01-15 09:53:22,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-01-15 09:53:22,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-01-15 09:53:22,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-01-15 09:53:22,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-01-15 09:53:22,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-01-15 09:53:22,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-01-15 09:53:22,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-01-15 09:53:22,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-01-15 09:53:22,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-01-15 09:53:22,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-01-15 09:53:22,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-01-15 09:53:22,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-01-15 09:53:22,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-01-15 09:53:22,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-01-15 09:53:22,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-01-15 09:53:22,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-01-15 09:53:22,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-01-15 09:53:22,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-01-15 09:53:22,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-01-15 09:53:22,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-01-15 09:53:22,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-01-15 09:53:22,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-01-15 09:53:22,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-01-15 09:53:22,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-01-15 09:53:22,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-01-15 09:53:22,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-01-15 09:53:22,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-01-15 09:53:22,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-01-15 09:53:22,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-01-15 09:53:22,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-01-15 09:53:22,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-01-15 09:53:22,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-01-15 09:53:22,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-01-15 09:53:22,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-01-15 09:53:22,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-01-15 09:53:22,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-01-15 09:53:22,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-01-15 09:53:22,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-01-15 09:53:22,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-01-15 09:53:22,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-01-15 09:53:22,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-01-15 09:53:22,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-01-15 09:53:22,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-01-15 09:53:22,239 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.5313429231318235E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.5313429231318235E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-15 09:53:22,239 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-01-15 09:53:22,239 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=4.024072417403382E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.5313429231318235E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.5313429231318235E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-01-15 09:53:22,482 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 375 ms to try 5000 different iterations.  Found a solution that moves 29 regions; Going from a computed imbalance of 4.024072417403382E-5 to a new imbalance of 6.235141079866791E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0013400504203456161); MoveCostFunction : (multiplier=7.0, imbalance=0.0032222222222222222); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-15 09:53:22,483 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1076039131:83 , srv1464355497:83 , srv243197673:83 , srv585791256:83 , srv639994593:83 , srv1002455356:84 , srv103782892:84 , srv108395837:84 , srv1090023692:84 , srv111758012:84 , srv1128581694:84 , srv1215966899:84 , srv1220855004:84 , srv1234859375:84 , srv1244076644:84 , srv1268162622:84 , srv1275501646:84 , srv1275654051:84 , srv1337337400:84 , srv134301949:84 , srv1347262222:84 , srv1354385606:84 , srv1382371943:84 , srv1394624167:84 , srv1453023226:84 , srv1455536078:84 , srv1510170785:84 , srv151321537:84 , srv1518215144:84 , srv1526447518:84 , srv1531936854:84 , srv1544156623:84 , srv1572174777:84 , srv1581413565:84 , srv1592835302:84 , srv1603522143:84 , srv1607122872:84 , srv165027652:84 , srv1673435253:84 , srv1682819961:84 , srv1683263250:84 , srv169956856:84 , srv1777309069:84 , srv1777398346:84 , srv1778503244:84 , srv1795339152:84 , srv1881541677:84 , srv1924103799:84 , srv1943920772:84 , srv1944362160:84 , srv1956089663:84 , srv1969616689:84 , srv1989411193:84 , srv1989736922:84 , srv2046372550:84 , srv2054646227:84 , srv2067476677:84 , srv2080426682:84 , srv2084218864:84 , srv2085272482:84 , srv21041165:84 , srv2118894492:84 , srv2122762435:84 , srv214182253:84 , srv281001750:84 , srv293098761:84 , srv294728847:84 , srv321369784:84 , srv322416354:84 , srv323829143:84 , srv369242239:84 , srv439937700:84 , srv453727220:84 , srv461830429:84 , srv498368520:84 , srv49909331:84 , srv551346764:84 , srv632405176:84 , srv646201600:84 , srv672703913:84 , srv718634315:84 , srv718637484:84 , srv796542873:84 , srv821648689:84 , srv839140648:84 , srv867475293:84 , srv877097957:84 , srv877280984:84 , srv890948332:84 , srv910435323:84 , srv922184644:84 , srv944980225:84 , srv960282199:84 , srv964713103:84 , srv1126918460:85 , srv1234820265:85 , srv156809153:85 , srv1816561880:85 , srv821392048:85 , srv855202663:684 }
2024-01-15 09:53:22,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-15 09:53:22,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1603522143=35, srv214182253=64, srv639994593=81, srv646201600=82, srv821392048=87, srv632405176=80, srv1275654051=15, srv1572174777=32, srv585791256=79, srv1394624167=21, srv1956089663=51, srv1777398346=43, srv498368520=76, srv890948332=94, srv243197673=65, srv1518215144=27, srv2080426682=58, srv461830429=75, srv1943920772=49, srv151321537=26, srv293098761=67, srv1275501646=14, srv21041165=61, srv1128581694=7, srv1510170785=25, srv169956856=41, srv281001750=66, srv1581413565=33, srv323829143=71, srv1455536078=23, srv1989736922=54, srv439937700=73, srv1126918460=6, srv1464355497=24, srv1881541677=47, srv1682819961=39, srv2084218864=59, srv1989411193=53, srv1607122872=36, srv1592835302=34, srv2118894492=62, srv718637484=85, srv1795339152=45, srv796542873=86, srv2067476677=57, srv453727220=74, srv1526447518=28, srv1531936854=29, srv1924103799=48, srv2122762435=63, srv551346764=78, srv718634315=84, srv111758012=5, srv1215966899=8, srv839140648=89, srv1268162622=13, srv2054646227=56, srv960282199=98, srv103782892=1, srv944980225=97, srv1002455356=0, srv294728847=68, srv369242239=72, srv1354385606=19, srv1382371943=20, srv49909331=77, srv1778503244=44, srv156809153=31, srv1944362160=50, srv1544156623=30, srv108395837=3, srv1090023692=4, srv134301949=17, srv1969616689=52, srv964713103=99, srv877280984=93, srv672703913=83, srv1244076644=12, srv1347262222=18, srv1453023226=22, srv855202663=90, srv1816561880=46, srv321369784=69, srv165027652=37, srv867475293=91, srv2085272482=60, srv1234820265=10, srv1076039131=2, srv1234859375=11, srv322416354=70, srv1337337400=16, srv1673435253=38, srv922184644=96, srv1220855004=9, srv1777309069=42, srv1683263250=40, srv877097957=92, srv2046372550=55, srv910435323=95, srv821648689=88} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-15 09:53:22,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-15 09:53:22,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-15 09:53:22,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-15 09:53:22,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-15 09:53:22,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-15 09:53:22,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-15 09:53:22,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-15 09:53:22,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-15 09:53:22,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-15 09:53:22,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-15 09:53:22,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-15 09:53:22,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-15 09:53:22,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-15 09:53:22,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-15 09:53:22,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-15 09:53:22,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-15 09:53:22,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-15 09:53:22,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-15 09:53:22,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-15 09:53:22,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-15 09:53:22,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-15 09:53:22,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-15 09:53:22,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-15 09:53:22,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-15 09:53:22,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-15 09:53:22,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-15 09:53:22,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-15 09:53:22,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-15 09:53:22,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-15 09:53:22,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-15 09:53:22,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-01-15 09:53:22,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-01-15 09:53:22,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-01-15 09:53:22,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-01-15 09:53:22,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-01-15 09:53:22,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-01-15 09:53:22,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-01-15 09:53:22,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-01-15 09:53:22,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-01-15 09:53:22,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-01-15 09:53:22,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-01-15 09:53:22,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-01-15 09:53:22,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-01-15 09:53:22,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-01-15 09:53:22,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-01-15 09:53:22,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-01-15 09:53:22,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-01-15 09:53:22,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-01-15 09:53:22,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-01-15 09:53:22,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-01-15 09:53:22,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-01-15 09:53:22,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-01-15 09:53:22,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-01-15 09:53:22,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-01-15 09:53:22,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-01-15 09:53:22,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-01-15 09:53:22,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-01-15 09:53:22,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-01-15 09:53:22,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-01-15 09:53:22,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-01-15 09:53:22,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-01-15 09:53:22,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-01-15 09:53:22,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-01-15 09:53:22,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-01-15 09:53:22,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-01-15 09:53:22,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-01-15 09:53:22,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-01-15 09:53:22,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-01-15 09:53:22,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-01-15 09:53:22,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-01-15 09:53:22,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-01-15 09:53:22,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-01-15 09:53:22,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-01-15 09:53:22,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-01-15 09:53:22,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-01-15 09:53:22,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-01-15 09:53:22,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-01-15 09:53:22,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-01-15 09:53:22,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-01-15 09:53:22,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-01-15 09:53:22,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-01-15 09:53:22,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-01-15 09:53:22,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-01-15 09:53:22,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-01-15 09:53:22,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-01-15 09:53:22,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-01-15 09:53:22,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-01-15 09:53:22,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-01-15 09:53:22,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-01-15 09:53:22,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-01-15 09:53:22,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-01-15 09:53:22,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-01-15 09:53:22,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-01-15 09:53:22,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-01-15 09:53:22,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-01-15 09:53:22,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-01-15 09:53:22,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-01-15 09:53:22,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-01-15 09:53:22,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-01-15 09:53:22,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-01-15 09:53:22,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-01-15 09:53:22,605 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=6.032078739728369E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0013400504203456161); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-15 09:53:22,605 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1076039131:83 , srv1464355497:83 , srv243197673:83 , srv585791256:83 , srv639994593:83 , srv1002455356:84 , srv103782892:84 , srv108395837:84 , srv1090023692:84 , srv111758012:84 , srv1128581694:84 , srv1215966899:84 , srv1220855004:84 , srv1234859375:84 , srv1244076644:84 , srv1268162622:84 , srv1275501646:84 , srv1275654051:84 , srv1337337400:84 , srv134301949:84 , srv1347262222:84 , srv1354385606:84 , srv1382371943:84 , srv1394624167:84 , srv1453023226:84 , srv1455536078:84 , srv1510170785:84 , srv151321537:84 , srv1518215144:84 , srv1526447518:84 , srv1531936854:84 , srv1544156623:84 , srv1572174777:84 , srv1581413565:84 , srv1592835302:84 , srv1603522143:84 , srv1607122872:84 , srv165027652:84 , srv1673435253:84 , srv1682819961:84 , srv1683263250:84 , srv169956856:84 , srv1777309069:84 , srv1777398346:84 , srv1778503244:84 , srv1795339152:84 , srv1881541677:84 , srv1924103799:84 , srv1943920772:84 , srv1944362160:84 , srv1956089663:84 , srv1969616689:84 , srv1989411193:84 , srv1989736922:84 , srv2046372550:84 , srv2054646227:84 , srv2067476677:84 , srv2080426682:84 , srv2084218864:84 , srv2085272482:84 , srv21041165:84 , srv2118894492:84 , srv2122762435:84 , srv214182253:84 , srv281001750:84 , srv293098761:84 , srv294728847:84 , srv321369784:84 , srv322416354:84 , srv323829143:84 , srv369242239:84 , srv439937700:84 , srv453727220:84 , srv461830429:84 , srv498368520:84 , srv49909331:84 , srv551346764:84 , srv632405176:84 , srv646201600:84 , srv672703913:84 , srv718634315:84 , srv718637484:84 , srv796542873:84 , srv821648689:84 , srv839140648:84 , srv867475293:84 , srv877097957:84 , srv877280984:84 , srv890948332:84 , srv910435323:84 , srv922184644:84 , srv944980225:84 , srv960282199:84 , srv964713103:84 , srv1126918460:85 , srv1234820265:85 , srv156809153:85 , srv1816561880:85 , srv821392048:85 , srv855202663:684 }
2024-01-15 09:53:22,630 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=295 (was 286) - SystemLoadAverage LEAK? -, ProcessCount=167 (was 167), AvailableMemoryMB=14512 (was 15130)
2024-01-15 09:53:22,638 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=295, ProcessCount=167, AvailableMemoryMB=14512
2024-01-15 09:53:22,639 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-01-15 09:53:22,639 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2024-01-15 09:53:22,648 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1194991513:84 , srv1291734118:84 , srv1391193351:84 , srv1400020659:84 , srv141509631:84 , srv1415218746:84 , srv1499591066:84 , srv1536208530:84 , srv1719165754:84 , srv1733148290:84 , srv1764998209:84 , srv1978390350:84 , srv2008662118:84 , srv2017685893:84 , srv2067026293:84 , srv2093719722:84 , srv2094367398:84 , srv2125447693:84 , srv230537537:84 , srv268466026:84 , srv325625408:84 , srv346723443:84 , srv468253603:84 , srv493802359:84 , srv522219342:84 , srv537433888:84 , srv619683309:84 , srv950599168:84 , srv955617832:84 , srv2096285979:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2024-01-15 09:53:22,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-15 09:53:22,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1733148290=9, srv1415218746=5, srv325625408=21, srv950599168=28, srv2067026293=14, srv1391193351=2, srv955617832=29, srv522219342=25, srv1536208530=7, srv1499591066=6, srv1764998209=10, srv2125447693=18, srv268466026=20, srv1719165754=8, srv2096285979=17, srv468253603=23, srv2094367398=16, srv230537537=19, srv493802359=24, srv2017685893=13, srv619683309=27, srv1978390350=11, srv2093719722=15, srv1291734118=1, srv1194991513=0, srv346723443=22, srv2008662118=12, srv537433888=26, srv141509631=4, srv1400020659=3} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-15 09:53:22,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-15 09:53:22,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-15 09:53:22,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-15 09:53:22,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-15 09:53:22,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-15 09:53:22,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-15 09:53:22,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-15 09:53:22,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-15 09:53:22,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-15 09:53:22,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-15 09:53:22,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-15 09:53:22,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-15 09:53:22,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-15 09:53:22,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-15 09:53:22,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-15 09:53:22,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-15 09:53:22,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-15 09:53:22,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-15 09:53:22,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-15 09:53:22,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-15 09:53:22,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-15 09:53:22,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-15 09:53:22,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-15 09:53:22,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-15 09:53:22,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-15 09:53:22,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-15 09:53:22,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-15 09:53:22,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-15 09:53:22,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-15 09:53:22,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-15 09:53:22,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-01-15 09:53:22,671 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1560220318058994, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-15 09:53:22,671 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912684663591911, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1560220318058994, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2024-01-15 09:55:22,682 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120013 ms to try 6254537 different iterations.  Found a solution that moves 1850 regions; Going from a computed imbalance of 0.9912684663591911 to a new imbalance of 4.317992290299789E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6851851851851852, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-15 09:55:22,719 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1194991513:90 , srv1291734118:90 , srv1391193351:90 , srv1400020659:90 , srv141509631:90 , srv1415218746:90 , srv1499591066:90 , srv1536208530:90 , srv1719165754:90 , srv1733148290:90 , srv1764998209:90 , srv1978390350:90 , srv2008662118:90 , srv2017685893:90 , srv2067026293:90 , srv2093719722:90 , srv2094367398:90 , srv2096285979:90 , srv2125447693:90 , srv230537537:90 , srv268466026:90 , srv325625408:90 , srv346723443:90 , srv468253603:90 , srv493802359:90 , srv522219342:90 , srv537433888:90 , srv619683309:90 , srv950599168:90 , srv955617832:90 }
2024-01-15 09:55:22,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-15 09:55:22,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1733148290=9, srv1415218746=5, srv325625408=21, srv950599168=28, srv2067026293=14, srv1391193351=2, srv955617832=29, srv522219342=25, srv1536208530=7, srv1499591066=6, srv1764998209=10, srv2125447693=18, srv268466026=20, srv1719165754=8, srv2096285979=17, srv468253603=23, srv2094367398=16, srv230537537=19, srv493802359=24, srv2017685893=13, srv619683309=27, srv1978390350=11, srv2093719722=15, srv1291734118=1, srv1194991513=0, srv346723443=22, srv2008662118=12, srv537433888=26, srv141509631=4, srv1400020659=3} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-15 09:55:22,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-15 09:55:22,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-15 09:55:22,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-15 09:55:22,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-15 09:55:22,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-15 09:55:22,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-15 09:55:22,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-15 09:55:22,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-15 09:55:22,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-15 09:55:22,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-15 09:55:22,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-15 09:55:22,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-15 09:55:22,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-15 09:55:22,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-15 09:55:22,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-15 09:55:22,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-15 09:55:22,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-15 09:55:22,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-15 09:55:22,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-15 09:55:22,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-15 09:55:22,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-15 09:55:22,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-15 09:55:22,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-15 09:55:22,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-15 09:55:22,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-15 09:55:22,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-15 09:55:22,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-15 09:55:22,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-15 09:55:22,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-15 09:55:22,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-15 09:55:22,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-01-15 09:55:22,874 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-15 09:55:22,874 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1194991513:90 , srv1291734118:90 , srv1391193351:90 , srv1400020659:90 , srv141509631:90 , srv1415218746:90 , srv1499591066:90 , srv1536208530:90 , srv1719165754:90 , srv1733148290:90 , srv1764998209:90 , srv1978390350:90 , srv2008662118:90 , srv2017685893:90 , srv2067026293:90 , srv2093719722:90 , srv2094367398:90 , srv2096285979:90 , srv2125447693:90 , srv230537537:90 , srv268466026:90 , srv325625408:90 , srv346723443:90 , srv468253603:90 , srv493802359:90 , srv522219342:90 , srv537433888:90 , srv619683309:90 , srv950599168:90 , srv955617832:90 }
2024-01-15 09:55:23,044 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=216 (was 295), ProcessCount=167 (was 167), AvailableMemoryMB=13944 (was 14512)