2024-02-11 08:53:11,243 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2024-02-11 08:53:11,570 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2024-02-11 08:53:11,671 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2024-02-11 08:53:11,734 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-02-11 08:53:11,741 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-02-11 08:53:11,753 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=276, ProcessCount=167, AvailableMemoryMB=3201
2024-02-11 08:53:11,756 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-02-11 08:53:11,756 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-02-11 08:53:11,885 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1011394663:84 , srv1023843341:84 , srv1037286128:84 , srv1056530290:84 , srv1061952467:84 , srv1079592893:84 , srv1098996858:84 , srv1121979321:84 , srv1148442319:84 , srv1178814787:84 , srv1191463751:84 , srv1197742626:84 , srv1206698966:84 , srv1220151015:84 , srv1225550479:84 , srv1258598369:84 , srv1262790479:84 , srv1267299436:84 , srv1299609237:84 , srv1328339685:84 , srv1333346751:84 , srv1342921754:84 , srv1391070141:84 , srv1435691219:84 , srv1447376454:84 , srv145671912:84 , srv1475032090:84 , srv1510509599:84 , srv1521520822:84 , srv1535437603:84 , srv1541976511:84 , srv1559296291:84 , srv1561307979:84 , srv1574914465:84 , srv1593832507:84 , srv1652350375:84 , srv1657045099:84 , srv1675558784:84 , srv1676089203:84 , srv1681736855:84 , srv1738402719:84 , srv1740416920:84 , srv1740450080:84 , srv1769854735:84 , srv1770715970:84 , srv1794329322:84 , srv1825081754:84 , srv1844374357:84 , srv1858882404:84 , srv1878544030:84 , srv1948822516:84 , srv1977086210:84 , srv200607295:84 , srv20188671:84 , srv2037908504:84 , srv2058691131:84 , srv20672533:84 , srv2142092973:84 , srv255212112:84 , srv267542445:84 , srv274059065:84 , srv356629517:84 , srv357377009:84 , srv383009719:84 , srv394578921:84 , srv399967985:84 , srv42830804:84 , srv461457756:84 , srv463150279:84 , srv495054059:84 , srv496791385:84 , srv502140118:84 , srv508712809:84 , srv529410407:84 , srv541603207:84 , srv573788868:84 , srv589295711:84 , srv597489687:84 , srv610439811:84 , srv61114630:84 , srv619137708:84 , srv621711897:84 , srv636918746:84 , srv65072371:84 , srv654978708:84 , srv683975402:84 , srv697418086:84 , srv739025599:84 , srv771800711:84 , srv775970449:84 , srv789971272:84 , srv831370006:84 , srv851300311:84 , srv852554288:84 , srv868346861:84 , srv900865760:84 , srv909896915:84 , srv962322707:84 , srv96645567:84 , srv1675632795:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2024-02-11 08:53:11,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-11 08:53:11,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1574914465=33, srv1328339685=19, srv394578921=65, srv1225550479=14, srv61114630=80, srv1561307979=32, srv868346861=95, srv909896915=97, srv1023843341=1, srv1178814787=9, srv851300311=93, srv1740450080=43, srv357377009=63, srv1794329322=46, srv852554288=94, srv1475032090=26, srv789971272=91, srv145671912=25, srv1011394663=0, srv1197742626=11, srv200607295=53, srv1258598369=15, srv508712809=73, srv1220151015=13, srv597489687=78, srv1676089203=39, srv771800711=89, srv274059065=61, srv900865760=96, srv529410407=74, srv502140118=72, srv267542445=60, srv1521520822=28, srv654978708=85, srv1206698966=12, srv1037286128=2, srv42830804=67, srv20188671=54, srv255212112=59, srv541603207=75, srv1061952467=4, srv1447376454=24, srv636918746=83, srv2058691131=56, srv383009719=64, srv1299609237=18, srv1191463751=10, srv1391070141=22, srv1098996858=6, srv1333346751=20, srv1738402719=41, srv619137708=81, srv1510509599=27, srv1740416920=42, srv1262790479=16, srv1267299436=17, srv1825081754=47, srv1559296291=31, srv1535437603=29, srv1541976511=30, srv1770715970=45, srv697418086=87, srv1056530290=3, srv610439811=79, srv1681736855=40, srv1858882404=49, srv1079592893=5, srv1948822516=51, srv356629517=62, srv683975402=86, srv461457756=68, srv1657045099=36, srv1844374357=48, srv1435691219=23, srv1121979321=7, srv1977086210=52, srv96645567=99, srv775970449=90, srv20672533=57, srv495054059=70, srv1652350375=35, srv463150279=69, srv2142092973=58, srv1148442319=8, srv2037908504=55, srv589295711=77, srv1342921754=21, srv496791385=71, srv962322707=98, srv739025599=88, srv831370006=92, srv621711897=82, srv399967985=66, srv1593832507=34, srv1675632795=38, srv573788868=76, srv65072371=84, srv1675558784=37, srv1769854735=44, srv1878544030=50} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-11 08:53:11,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-11 08:53:11,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-11 08:53:11,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-11 08:53:11,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-11 08:53:11,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-11 08:53:11,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-11 08:53:11,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-11 08:53:11,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-11 08:53:11,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-11 08:53:11,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-11 08:53:11,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-11 08:53:11,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-11 08:53:11,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-11 08:53:11,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-11 08:53:11,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-11 08:53:11,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-11 08:53:11,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-11 08:53:11,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-11 08:53:11,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-11 08:53:11,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-11 08:53:11,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-11 08:53:11,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-11 08:53:11,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-11 08:53:11,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-11 08:53:11,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-11 08:53:11,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-11 08:53:11,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-11 08:53:11,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-11 08:53:11,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-11 08:53:11,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-11 08:53:11,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-02-11 08:53:11,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-02-11 08:53:11,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-02-11 08:53:11,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-02-11 08:53:11,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-02-11 08:53:11,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-02-11 08:53:11,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-02-11 08:53:11,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-02-11 08:53:11,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-02-11 08:53:11,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-02-11 08:53:11,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-02-11 08:53:11,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-02-11 08:53:11,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-02-11 08:53:11,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-02-11 08:53:11,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-02-11 08:53:11,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-02-11 08:53:11,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-02-11 08:53:11,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-02-11 08:53:11,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-02-11 08:53:11,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-02-11 08:53:11,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-02-11 08:53:11,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-02-11 08:53:11,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-02-11 08:53:11,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-02-11 08:53:11,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-02-11 08:53:11,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-02-11 08:53:11,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-02-11 08:53:11,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-02-11 08:53:11,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-02-11 08:53:11,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-02-11 08:53:11,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-02-11 08:53:11,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-02-11 08:53:11,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-02-11 08:53:11,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-02-11 08:53:11,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-02-11 08:53:11,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-02-11 08:53:11,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-02-11 08:53:11,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-02-11 08:53:11,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-02-11 08:53:11,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-02-11 08:53:11,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-02-11 08:53:11,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-02-11 08:53:11,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-02-11 08:53:11,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-02-11 08:53:11,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-02-11 08:53:11,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-02-11 08:53:11,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-02-11 08:53:11,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-02-11 08:53:11,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-02-11 08:53:11,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-02-11 08:53:11,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-02-11 08:53:11,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-02-11 08:53:11,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-02-11 08:53:11,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-02-11 08:53:11,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-02-11 08:53:11,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-02-11 08:53:11,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-02-11 08:53:11,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-02-11 08:53:11,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-02-11 08:53:11,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-02-11 08:53:11,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-02-11 08:53:11,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-02-11 08:53:11,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-02-11 08:53:11,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-02-11 08:53:11,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-02-11 08:53:11,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-02-11 08:53:11,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-02-11 08:53:11,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-02-11 08:53:11,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-02-11 08:53:11,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-02-11 08:53:11,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-02-11 08:53:12,223 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-11 08:53:12,223 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-02-11 08:53:12,224 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-02-11 08:53:12,658 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 639 ms to try 5000 different iterations.  Found a solution that moves 4658 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1789436125577619. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.037872726787801195, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05693637538297324, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5175555555555555, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.037872726787801195, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17416666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24075, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-11 08:53:12,672 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1769854735:80 , srv1148442319:82 , srv1652350375:82 , srv20672533:82 , srv508712809:82 , srv900865760:82 , srv96645567:82 , srv1121979321:83 , srv1328339685:83 , srv1333346751:83 , srv1510509599:83 , srv255212112:83 , srv463150279:83 , srv909896915:83 , srv1023843341:84 , srv1435691219:84 , srv1535437603:84 , srv1675558784:84 , srv2058691131:84 , srv495054059:84 , srv541603207:84 , srv619137708:84 , srv1079592893:85 , srv1197742626:85 , srv1258598369:85 , srv1262790479:85 , srv1574914465:85 , srv1657045099:85 , srv1825081754:85 , srv1844374357:85 , srv383009719:85 , srv496791385:85 , srv61114630:85 , srv621711897:85 , srv683975402:85 , srv868346861:85 , srv1037286128:86 , srv1391070141:86 , srv145671912:86 , srv1676089203:86 , srv200607295:86 , srv394578921:86 , srv42830804:86 , srv502140118:86 , srv573788868:86 , srv589295711:86 , srv636918746:86 , srv697418086:86 , srv789971272:86 , srv831370006:86 , srv1191463751:87 , srv1220151015:87 , srv1299609237:87 , srv1342921754:87 , srv1593832507:87 , srv1740450080:87 , srv1948822516:87 , srv1977086210:87 , srv2037908504:87 , srv2142092973:87 , srv274059065:87 , srv461457756:87 , srv597489687:87 , srv771800711:87 , srv852554288:87 , srv1011394663:88 , srv1267299436:88 , srv1447376454:88 , srv1475032090:88 , srv1559296291:88 , srv1561307979:88 , srv1681736855:88 , srv1740416920:88 , srv1878544030:88 , srv356629517:88 , srv357377009:88 , srv399967985:88 , srv610439811:88 , srv65072371:88 , srv654978708:88 , srv962322707:88 , srv1098996858:89 , srv1178814787:89 , srv1206698966:89 , srv1225550479:89 , srv1521520822:89 , srv1770715970:89 , srv20188671:89 , srv529410407:89 , srv775970449:89 , srv1056530290:90 , srv1738402719:90 , srv1858882404:90 , srv851300311:90 , srv1541976511:92 , srv1061952467:95 , srv267542445:97 , srv739025599:97 , srv1794329322:100 , srv1675632795:426 }
2024-02-11 08:53:12,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-11 08:53:12,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1574914465=33, srv1328339685=19, srv394578921=65, srv1225550479=14, srv61114630=80, srv1561307979=32, srv868346861=95, srv909896915=97, srv1023843341=1, srv1178814787=9, srv851300311=93, srv1740450080=43, srv357377009=63, srv1794329322=46, srv852554288=94, srv1475032090=26, srv789971272=91, srv145671912=25, srv1011394663=0, srv1197742626=11, srv200607295=53, srv1258598369=15, srv508712809=73, srv1220151015=13, srv597489687=78, srv1676089203=39, srv771800711=89, srv274059065=61, srv900865760=96, srv529410407=74, srv502140118=72, srv267542445=60, srv1521520822=28, srv654978708=85, srv1206698966=12, srv1037286128=2, srv42830804=67, srv20188671=54, srv255212112=59, srv541603207=75, srv1061952467=4, srv1447376454=24, srv636918746=83, srv2058691131=56, srv383009719=64, srv1299609237=18, srv1191463751=10, srv1391070141=22, srv1098996858=6, srv1333346751=20, srv1738402719=41, srv619137708=81, srv1510509599=27, srv1740416920=42, srv1262790479=16, srv1267299436=17, srv1825081754=47, srv1559296291=31, srv1535437603=29, srv1541976511=30, srv1770715970=45, srv697418086=87, srv1056530290=3, srv610439811=79, srv1681736855=40, srv1858882404=49, srv1079592893=5, srv1948822516=51, srv356629517=62, srv683975402=86, srv461457756=68, srv1657045099=36, srv1844374357=48, srv1435691219=23, srv1121979321=7, srv1977086210=52, srv96645567=99, srv775970449=90, srv20672533=57, srv495054059=70, srv1652350375=35, srv463150279=69, srv2142092973=58, srv1148442319=8, srv2037908504=55, srv589295711=77, srv1342921754=21, srv496791385=71, srv962322707=98, srv739025599=88, srv831370006=92, srv621711897=82, srv399967985=66, srv1593832507=34, srv1675632795=38, srv573788868=76, srv65072371=84, srv1675558784=37, srv1769854735=44, srv1878544030=50} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-11 08:53:12,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-11 08:53:12,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-11 08:53:12,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-11 08:53:12,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-11 08:53:12,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-11 08:53:12,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-11 08:53:12,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-11 08:53:12,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-11 08:53:12,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-11 08:53:12,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-11 08:53:12,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-11 08:53:12,694 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-11 08:53:12,694 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-11 08:53:12,694 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-11 08:53:12,694 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-11 08:53:12,694 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-11 08:53:12,694 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-11 08:53:12,694 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-11 08:53:12,694 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-11 08:53:12,695 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-11 08:53:12,695 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-11 08:53:12,695 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-11 08:53:12,695 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-11 08:53:12,695 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-11 08:53:12,695 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-11 08:53:12,695 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-11 08:53:12,695 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-11 08:53:12,695 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-11 08:53:12,695 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-11 08:53:12,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-11 08:53:12,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-02-11 08:53:12,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-02-11 08:53:12,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-02-11 08:53:12,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-02-11 08:53:12,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-02-11 08:53:12,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-02-11 08:53:12,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-02-11 08:53:12,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-02-11 08:53:12,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-02-11 08:53:12,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-02-11 08:53:12,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-02-11 08:53:12,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-02-11 08:53:12,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-02-11 08:53:12,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-02-11 08:53:12,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-02-11 08:53:12,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-02-11 08:53:12,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-02-11 08:53:12,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-02-11 08:53:12,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-02-11 08:53:12,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-02-11 08:53:12,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-02-11 08:53:12,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-02-11 08:53:12,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-02-11 08:53:12,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-02-11 08:53:12,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-02-11 08:53:12,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-02-11 08:53:12,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-02-11 08:53:12,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-02-11 08:53:12,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-02-11 08:53:12,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-02-11 08:53:12,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-02-11 08:53:12,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-02-11 08:53:12,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-02-11 08:53:12,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-02-11 08:53:12,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-02-11 08:53:12,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-02-11 08:53:12,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-02-11 08:53:12,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-02-11 08:53:12,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-02-11 08:53:12,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-02-11 08:53:12,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-02-11 08:53:12,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-02-11 08:53:12,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-02-11 08:53:12,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-02-11 08:53:12,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-02-11 08:53:12,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-02-11 08:53:12,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-02-11 08:53:12,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-02-11 08:53:12,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-02-11 08:53:12,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-02-11 08:53:12,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-02-11 08:53:12,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-02-11 08:53:12,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-02-11 08:53:12,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-02-11 08:53:12,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-02-11 08:53:12,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-02-11 08:53:12,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-02-11 08:53:12,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-02-11 08:53:12,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-02-11 08:53:12,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-02-11 08:53:12,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-02-11 08:53:12,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-02-11 08:53:12,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-02-11 08:53:12,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-02-11 08:53:12,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-02-11 08:53:12,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-02-11 08:53:12,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-02-11 08:53:12,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-02-11 08:53:12,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-02-11 08:53:12,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-02-11 08:53:12,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-02-11 08:53:12,890 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.037872726787801195, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05693637538297324, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.037872726787801195, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17416666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24075, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-11 08:53:12,890 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-02-11 08:53:12,890 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17891099654464585, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.037872726787801195, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05693637538297324, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.037872726787801195, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17416666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24075, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-02-11 08:53:13,176 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 448 ms to try 5000 different iterations.  Found a solution that moves 2650 regions; Going from a computed imbalance of 0.17891099654464585 to a new imbalance of 0.014843372366770951. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02028233194558177); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03335689403030573, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29444444444444445, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02028233194558177); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011583333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04608333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-11 08:53:13,187 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1794329322:72 , srv739025599:73 , srv1061952467:77 , srv1738402719:79 , srv267542445:79 , srv1056530290:81 , srv1178814787:81 , srv1475032090:81 , srv1541976511:81 , srv1225550479:82 , srv1858882404:82 , srv1342921754:83 , srv1593832507:83 , srv1740416920:83 , srv1770715970:83 , srv20188671:83 , srv597489687:83 , srv851300311:83 , srv1206698966:84 , srv1447376454:84 , srv1559296291:84 , srv1825081754:84 , srv200607295:84 , srv274059065:84 , srv356629517:84 , srv357377009:84 , srv529410407:84 , srv610439811:84 , srv654978708:84 , srv697418086:84 , srv775970449:84 , srv852554288:84 , srv1037286128:85 , srv1191463751:85 , srv1262790479:85 , srv1267299436:85 , srv1657045099:85 , srv1740450080:85 , srv1878544030:85 , srv2037908504:85 , srv2142092973:85 , srv394578921:85 , srv461457756:85 , srv573788868:85 , srv831370006:85 , srv1079592893:86 , srv1098996858:86 , srv1299609237:86 , srv1521520822:86 , srv1535437603:86 , srv1948822516:86 , srv1977086210:86 , srv2058691131:86 , srv383009719:86 , srv496791385:86 , srv589295711:86 , srv683975402:86 , srv771800711:86 , srv962322707:86 , srv1011394663:87 , srv1220151015:87 , srv1391070141:87 , srv145671912:87 , srv1574914465:87 , srv399967985:87 , srv502140118:87 , srv621711897:87 , srv636918746:87 , srv65072371:87 , srv1148442319:88 , srv1197742626:88 , srv1258598369:88 , srv1561307979:88 , srv1675558784:88 , srv1676089203:88 , srv1844374357:88 , srv20672533:88 , srv255212112:88 , srv463150279:88 , srv61114630:88 , srv789971272:88 , srv868346861:88 , srv1333346751:89 , srv1510509599:89 , srv1681736855:89 , srv42830804:89 , srv495054059:89 , srv541603207:89 , srv619137708:89 , srv900865760:89 , srv96645567:89 , srv1023843341:90 , srv1121979321:90 , srv1328339685:90 , srv508712809:90 , srv909896915:90 , srv1435691219:91 , srv1652350375:91 , srv1769854735:91 , srv1675632795:528 }
2024-02-11 08:53:13,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-11 08:53:13,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1574914465=33, srv1328339685=19, srv394578921=65, srv1225550479=14, srv61114630=80, srv1561307979=32, srv868346861=95, srv909896915=97, srv1023843341=1, srv1178814787=9, srv851300311=93, srv1740450080=43, srv357377009=63, srv1794329322=46, srv852554288=94, srv1475032090=26, srv789971272=91, srv145671912=25, srv1011394663=0, srv1197742626=11, srv200607295=53, srv1258598369=15, srv508712809=73, srv1220151015=13, srv597489687=78, srv1676089203=39, srv771800711=89, srv274059065=61, srv900865760=96, srv529410407=74, srv502140118=72, srv267542445=60, srv1521520822=28, srv654978708=85, srv1206698966=12, srv1037286128=2, srv42830804=67, srv20188671=54, srv255212112=59, srv541603207=75, srv1061952467=4, srv1447376454=24, srv636918746=83, srv2058691131=56, srv383009719=64, srv1299609237=18, srv1191463751=10, srv1391070141=22, srv1098996858=6, srv1333346751=20, srv1738402719=41, srv619137708=81, srv1510509599=27, srv1740416920=42, srv1262790479=16, srv1267299436=17, srv1825081754=47, srv1559296291=31, srv1535437603=29, srv1541976511=30, srv1770715970=45, srv697418086=87, srv1056530290=3, srv610439811=79, srv1681736855=40, srv1858882404=49, srv1079592893=5, srv1948822516=51, srv356629517=62, srv683975402=86, srv461457756=68, srv1657045099=36, srv1844374357=48, srv1435691219=23, srv1121979321=7, srv1977086210=52, srv96645567=99, srv775970449=90, srv20672533=57, srv495054059=70, srv1652350375=35, srv463150279=69, srv2142092973=58, srv1148442319=8, srv2037908504=55, srv589295711=77, srv1342921754=21, srv496791385=71, srv962322707=98, srv739025599=88, srv831370006=92, srv621711897=82, srv399967985=66, srv1593832507=34, srv1675632795=38, srv573788868=76, srv65072371=84, srv1675558784=37, srv1769854735=44, srv1878544030=50} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-11 08:53:13,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-11 08:53:13,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-11 08:53:13,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-11 08:53:13,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-11 08:53:13,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-11 08:53:13,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-11 08:53:13,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-11 08:53:13,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-11 08:53:13,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-11 08:53:13,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-11 08:53:13,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-11 08:53:13,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-11 08:53:13,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-11 08:53:13,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-11 08:53:13,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-11 08:53:13,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-11 08:53:13,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-11 08:53:13,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-11 08:53:13,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-11 08:53:13,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-11 08:53:13,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-11 08:53:13,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-11 08:53:13,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-11 08:53:13,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-11 08:53:13,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-11 08:53:13,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-11 08:53:13,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-11 08:53:13,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-11 08:53:13,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-11 08:53:13,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-11 08:53:13,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-02-11 08:53:13,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-02-11 08:53:13,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-02-11 08:53:13,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-02-11 08:53:13,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-02-11 08:53:13,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-02-11 08:53:13,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-02-11 08:53:13,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-02-11 08:53:13,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-02-11 08:53:13,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-02-11 08:53:13,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-02-11 08:53:13,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-02-11 08:53:13,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-02-11 08:53:13,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-02-11 08:53:13,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-02-11 08:53:13,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-02-11 08:53:13,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-02-11 08:53:13,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-02-11 08:53:13,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-02-11 08:53:13,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-02-11 08:53:13,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-02-11 08:53:13,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-02-11 08:53:13,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-02-11 08:53:13,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-02-11 08:53:13,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-02-11 08:53:13,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-02-11 08:53:13,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-02-11 08:53:13,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-02-11 08:53:13,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-02-11 08:53:13,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-02-11 08:53:13,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-02-11 08:53:13,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-02-11 08:53:13,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-02-11 08:53:13,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-02-11 08:53:13,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-02-11 08:53:13,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-02-11 08:53:13,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-02-11 08:53:13,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-02-11 08:53:13,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-02-11 08:53:13,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-02-11 08:53:13,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-02-11 08:53:13,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-02-11 08:53:13,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-02-11 08:53:13,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-02-11 08:53:13,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-02-11 08:53:13,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-02-11 08:53:13,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-02-11 08:53:13,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-02-11 08:53:13,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-02-11 08:53:13,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-02-11 08:53:13,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-02-11 08:53:13,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-02-11 08:53:13,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-02-11 08:53:13,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-02-11 08:53:13,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-02-11 08:53:13,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-02-11 08:53:13,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-02-11 08:53:13,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-02-11 08:53:13,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-02-11 08:53:13,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-02-11 08:53:13,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-02-11 08:53:13,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-02-11 08:53:13,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-02-11 08:53:13,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-02-11 08:53:13,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-02-11 08:53:13,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-02-11 08:53:13,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-02-11 08:53:13,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-02-11 08:53:13,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-02-11 08:53:13,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-02-11 08:53:13,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-02-11 08:53:13,365 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02028233194558177); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03335689403030573, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02028233194558177); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011583333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04608333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-11 08:53:13,366 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-02-11 08:53:13,366 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.014824816670172094, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02028233194558177); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03335689403030573, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02028233194558177); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011583333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04608333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-02-11 08:53:13,737 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 477 ms to try 5000 different iterations.  Found a solution that moves 987 regions; Going from a computed imbalance of 0.014824816670172094 to a new imbalance of 9.284421673327911E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006091950371931928); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009870087677015036); MoveCostFunction : (multiplier=7.0, imbalance=0.10966666666666666, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006091950371931928); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.0E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-11 08:53:13,740 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1681736855:80 , srv1561307979:81 , srv267542445:81 , srv1098996858:82 , srv42830804:82 , srv65072371:82 , srv1011394663:83 , srv1023843341:83 , srv1206698966:83 , srv1435691219:83 , srv1521520822:83 , srv1676089203:83 , srv399967985:83 , srv1121979321:84 , srv1197742626:84 , srv1220151015:84 , srv1258598369:84 , srv1328339685:84 , srv1391070141:84 , srv1541976511:84 , srv1652350375:84 , srv1844374357:84 , srv1878544030:84 , srv1948822516:84 , srv495054059:84 , srv529410407:84 , srv61114630:84 , srv789971272:84 , srv909896915:84 , srv962322707:84 , srv1061952467:85 , srv1191463751:85 , srv1267299436:85 , srv1299609237:85 , srv1333346751:85 , srv145671912:85 , srv1510509599:85 , srv1675558784:85 , srv1740450080:85 , srv1794329322:85 , srv1977086210:85 , srv2142092973:85 , srv357377009:85 , srv461457756:85 , srv502140118:85 , srv541603207:85 , srv610439811:85 , srv619137708:85 , srv636918746:85 , srv771800711:85 , srv775970449:85 , srv851300311:85 , srv868346861:85 , srv1056530290:86 , srv1225550479:86 , srv1559296291:86 , srv1574914465:86 , srv1740416920:86 , srv1770715970:86 , srv1858882404:86 , srv20188671:86 , srv2037908504:86 , srv255212112:86 , srv356629517:86 , srv394578921:86 , srv496791385:86 , srv508712809:86 , srv573788868:86 , srv589295711:86 , srv654978708:86 , srv683975402:86 , srv831370006:86 , srv852554288:86 , srv96645567:86 , srv1037286128:87 , srv1079592893:87 , srv1178814787:87 , srv1262790479:87 , srv1447376454:87 , srv1593832507:87 , srv1769854735:87 , srv200607295:87 , srv2058691131:87 , srv274059065:87 , srv383009719:87 , srv463150279:87 , srv597489687:87 , srv621711897:87 , srv697418086:87 , srv739025599:87 , srv900865760:87 , srv1148442319:88 , srv1342921754:88 , srv1475032090:88 , srv1535437603:88 , srv1657045099:88 , srv1825081754:88 , srv20672533:88 , srv1738402719:89 , srv1675632795:558 }
2024-02-11 08:53:13,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-11 08:53:13,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1574914465=33, srv1328339685=19, srv394578921=65, srv1225550479=14, srv61114630=80, srv1561307979=32, srv868346861=95, srv909896915=97, srv1023843341=1, srv1178814787=9, srv851300311=93, srv1740450080=43, srv357377009=63, srv1794329322=46, srv852554288=94, srv1475032090=26, srv789971272=91, srv145671912=25, srv1011394663=0, srv1197742626=11, srv200607295=53, srv1258598369=15, srv508712809=73, srv1220151015=13, srv597489687=78, srv1676089203=39, srv771800711=89, srv274059065=61, srv900865760=96, srv529410407=74, srv502140118=72, srv267542445=60, srv1521520822=28, srv654978708=85, srv1206698966=12, srv1037286128=2, srv42830804=67, srv20188671=54, srv255212112=59, srv541603207=75, srv1061952467=4, srv1447376454=24, srv636918746=83, srv2058691131=56, srv383009719=64, srv1299609237=18, srv1191463751=10, srv1391070141=22, srv1098996858=6, srv1333346751=20, srv1738402719=41, srv619137708=81, srv1510509599=27, srv1740416920=42, srv1262790479=16, srv1267299436=17, srv1825081754=47, srv1559296291=31, srv1535437603=29, srv1541976511=30, srv1770715970=45, srv697418086=87, srv1056530290=3, srv610439811=79, srv1681736855=40, srv1858882404=49, srv1079592893=5, srv1948822516=51, srv356629517=62, srv683975402=86, srv461457756=68, srv1657045099=36, srv1844374357=48, srv1435691219=23, srv1121979321=7, srv1977086210=52, srv96645567=99, srv775970449=90, srv20672533=57, srv495054059=70, srv1652350375=35, srv463150279=69, srv2142092973=58, srv1148442319=8, srv2037908504=55, srv589295711=77, srv1342921754=21, srv496791385=71, srv962322707=98, srv739025599=88, srv831370006=92, srv621711897=82, srv399967985=66, srv1593832507=34, srv1675632795=38, srv573788868=76, srv65072371=84, srv1675558784=37, srv1769854735=44, srv1878544030=50} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-11 08:53:13,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-11 08:53:13,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-11 08:53:13,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-11 08:53:13,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-11 08:53:13,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-11 08:53:13,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-11 08:53:13,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-11 08:53:13,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-11 08:53:13,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-11 08:53:13,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-11 08:53:13,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-11 08:53:13,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-11 08:53:13,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-11 08:53:13,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-11 08:53:13,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-11 08:53:13,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-11 08:53:13,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-11 08:53:13,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-11 08:53:13,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-11 08:53:13,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-11 08:53:13,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-11 08:53:13,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-11 08:53:13,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-11 08:53:13,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-11 08:53:13,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-11 08:53:13,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-11 08:53:13,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-11 08:53:13,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-11 08:53:13,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-11 08:53:13,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-11 08:53:13,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-02-11 08:53:13,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-02-11 08:53:13,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-02-11 08:53:13,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-02-11 08:53:13,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-02-11 08:53:13,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-02-11 08:53:13,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-02-11 08:53:13,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-02-11 08:53:13,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-02-11 08:53:13,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-02-11 08:53:13,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-02-11 08:53:13,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-02-11 08:53:13,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-02-11 08:53:13,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-02-11 08:53:13,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-02-11 08:53:13,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-02-11 08:53:13,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-02-11 08:53:13,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-02-11 08:53:13,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-02-11 08:53:13,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-02-11 08:53:13,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-02-11 08:53:13,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-02-11 08:53:13,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-02-11 08:53:13,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-02-11 08:53:13,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-02-11 08:53:13,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-02-11 08:53:13,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-02-11 08:53:13,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-02-11 08:53:13,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-02-11 08:53:13,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-02-11 08:53:13,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-02-11 08:53:13,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-02-11 08:53:13,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-02-11 08:53:13,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-02-11 08:53:13,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-02-11 08:53:13,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-02-11 08:53:13,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-02-11 08:53:13,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-02-11 08:53:13,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-02-11 08:53:13,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-02-11 08:53:13,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-02-11 08:53:13,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-02-11 08:53:13,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-02-11 08:53:13,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-02-11 08:53:13,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-02-11 08:53:13,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-02-11 08:53:13,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-02-11 08:53:13,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-02-11 08:53:13,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-02-11 08:53:13,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-02-11 08:53:13,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-02-11 08:53:13,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-02-11 08:53:13,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-02-11 08:53:13,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-02-11 08:53:13,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-02-11 08:53:13,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-02-11 08:53:13,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-02-11 08:53:13,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-02-11 08:53:13,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-02-11 08:53:13,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-02-11 08:53:13,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-02-11 08:53:13,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-02-11 08:53:13,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-02-11 08:53:13,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-02-11 08:53:13,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-02-11 08:53:13,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-02-11 08:53:13,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-02-11 08:53:13,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-02-11 08:53:13,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-02-11 08:53:13,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-02-11 08:53:13,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-02-11 08:53:13,927 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006091950371931928); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009870087677015036); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006091950371931928); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.0E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-11 08:53:13,927 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-02-11 08:53:13,928 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=9.215310456184249E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006091950371931928); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009870087677015036); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006091950371931928); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.0E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-02-11 08:53:14,320 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 534 ms to try 5000 different iterations.  Found a solution that moves 255 regions; Going from a computed imbalance of 9.215310456184249E-4 to a new imbalance of 2.745530319264674E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.028333333333333332, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-11 08:53:14,321 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv356629517:83 , srv399967985:83 , srv621711897:83 , srv775970449:83 , srv1011394663:84 , srv1037286128:84 , srv1079592893:84 , srv1148442319:84 , srv1267299436:84 , srv1299609237:84 , srv1342921754:84 , srv1447376454:84 , srv145671912:84 , srv1521520822:84 , srv1535437603:84 , srv1541976511:84 , srv1559296291:84 , srv1593832507:84 , srv1657045099:84 , srv1738402719:84 , srv1740416920:84 , srv1769854735:84 , srv1770715970:84 , srv1844374357:84 , srv1977086210:84 , srv20188671:84 , srv20672533:84 , srv274059065:84 , srv357377009:84 , srv383009719:84 , srv463150279:84 , srv495054059:84 , srv502140118:84 , srv508712809:84 , srv589295711:84 , srv619137708:84 , srv636918746:84 , srv654978708:84 , srv771800711:84 , srv789971272:84 , srv851300311:84 , srv852554288:84 , srv900865760:84 , srv962322707:84 , srv1023843341:85 , srv1056530290:85 , srv1061952467:85 , srv1098996858:85 , srv1121979321:85 , srv1178814787:85 , srv1191463751:85 , srv1197742626:85 , srv1220151015:85 , srv1225550479:85 , srv1258598369:85 , srv1262790479:85 , srv1328339685:85 , srv1391070141:85 , srv1435691219:85 , srv1475032090:85 , srv1510509599:85 , srv1561307979:85 , srv1574914465:85 , srv1652350375:85 , srv1675558784:85 , srv1676089203:85 , srv1681736855:85 , srv1740450080:85 , srv1794329322:85 , srv1825081754:85 , srv1858882404:85 , srv1878544030:85 , srv1948822516:85 , srv200607295:85 , srv2037908504:85 , srv2058691131:85 , srv2142092973:85 , srv255212112:85 , srv394578921:85 , srv42830804:85 , srv461457756:85 , srv496791385:85 , srv529410407:85 , srv541603207:85 , srv573788868:85 , srv597489687:85 , srv610439811:85 , srv61114630:85 , srv65072371:85 , srv683975402:85 , srv697418086:85 , srv739025599:85 , srv831370006:85 , srv868346861:85 , srv909896915:85 , srv96645567:85 , srv1206698966:86 , srv1333346751:86 , srv267542445:86 , srv1675632795:630 }
2024-02-11 08:53:14,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-11 08:53:14,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1574914465=33, srv1328339685=19, srv394578921=65, srv1225550479=14, srv61114630=80, srv1561307979=32, srv868346861=95, srv909896915=97, srv1023843341=1, srv1178814787=9, srv851300311=93, srv1740450080=43, srv357377009=63, srv1794329322=46, srv852554288=94, srv1475032090=26, srv789971272=91, srv145671912=25, srv1011394663=0, srv1197742626=11, srv200607295=53, srv1258598369=15, srv508712809=73, srv1220151015=13, srv597489687=78, srv1676089203=39, srv771800711=89, srv274059065=61, srv900865760=96, srv529410407=74, srv502140118=72, srv267542445=60, srv1521520822=28, srv654978708=85, srv1206698966=12, srv1037286128=2, srv42830804=67, srv20188671=54, srv255212112=59, srv541603207=75, srv1061952467=4, srv1447376454=24, srv636918746=83, srv2058691131=56, srv383009719=64, srv1299609237=18, srv1191463751=10, srv1391070141=22, srv1098996858=6, srv1333346751=20, srv1738402719=41, srv619137708=81, srv1510509599=27, srv1740416920=42, srv1262790479=16, srv1267299436=17, srv1825081754=47, srv1559296291=31, srv1535437603=29, srv1541976511=30, srv1770715970=45, srv697418086=87, srv1056530290=3, srv610439811=79, srv1681736855=40, srv1858882404=49, srv1079592893=5, srv1948822516=51, srv356629517=62, srv683975402=86, srv461457756=68, srv1657045099=36, srv1844374357=48, srv1435691219=23, srv1121979321=7, srv1977086210=52, srv96645567=99, srv775970449=90, srv20672533=57, srv495054059=70, srv1652350375=35, srv463150279=69, srv2142092973=58, srv1148442319=8, srv2037908504=55, srv589295711=77, srv1342921754=21, srv496791385=71, srv962322707=98, srv739025599=88, srv831370006=92, srv621711897=82, srv399967985=66, srv1593832507=34, srv1675632795=38, srv573788868=76, srv65072371=84, srv1675558784=37, srv1769854735=44, srv1878544030=50} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-11 08:53:14,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-11 08:53:14,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-11 08:53:14,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-11 08:53:14,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-11 08:53:14,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-11 08:53:14,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-11 08:53:14,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-11 08:53:14,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-11 08:53:14,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-11 08:53:14,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-11 08:53:14,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-11 08:53:14,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-11 08:53:14,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-11 08:53:14,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-11 08:53:14,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-11 08:53:14,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-11 08:53:14,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-11 08:53:14,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-11 08:53:14,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-11 08:53:14,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-11 08:53:14,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-11 08:53:14,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-11 08:53:14,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-11 08:53:14,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-11 08:53:14,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-11 08:53:14,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-11 08:53:14,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-11 08:53:14,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-11 08:53:14,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-11 08:53:14,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-11 08:53:14,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-02-11 08:53:14,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-02-11 08:53:14,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-02-11 08:53:14,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-02-11 08:53:14,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-02-11 08:53:14,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-02-11 08:53:14,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-02-11 08:53:14,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-02-11 08:53:14,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-02-11 08:53:14,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-02-11 08:53:14,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-02-11 08:53:14,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-02-11 08:53:14,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-02-11 08:53:14,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-02-11 08:53:14,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-02-11 08:53:14,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-02-11 08:53:14,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-02-11 08:53:14,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-02-11 08:53:14,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-02-11 08:53:14,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-02-11 08:53:14,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-02-11 08:53:14,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-02-11 08:53:14,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-02-11 08:53:14,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-02-11 08:53:14,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-02-11 08:53:14,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-02-11 08:53:14,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-02-11 08:53:14,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-02-11 08:53:14,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-02-11 08:53:14,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-02-11 08:53:14,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-02-11 08:53:14,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-02-11 08:53:14,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-02-11 08:53:14,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-02-11 08:53:14,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-02-11 08:53:14,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-02-11 08:53:14,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-02-11 08:53:14,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-02-11 08:53:14,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-02-11 08:53:14,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-02-11 08:53:14,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-02-11 08:53:14,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-02-11 08:53:14,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-02-11 08:53:14,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-02-11 08:53:14,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-02-11 08:53:14,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-02-11 08:53:14,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-02-11 08:53:14,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-02-11 08:53:14,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-02-11 08:53:14,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-02-11 08:53:14,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-02-11 08:53:14,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-02-11 08:53:14,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-02-11 08:53:14,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-02-11 08:53:14,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-02-11 08:53:14,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-02-11 08:53:14,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-02-11 08:53:14,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-02-11 08:53:14,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-02-11 08:53:14,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-02-11 08:53:14,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-02-11 08:53:14,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-02-11 08:53:14,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-02-11 08:53:14,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-02-11 08:53:14,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-02-11 08:53:14,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-02-11 08:53:14,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-02-11 08:53:14,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-02-11 08:53:14,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-02-11 08:53:14,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-02-11 08:53:14,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-02-11 08:53:14,443 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-11 08:53:14,443 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-02-11 08:53:14,443 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=2.566975502936061E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-02-11 08:53:14,683 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 333 ms to try 5000 different iterations.  Found a solution that moves 26 regions; Going from a computed imbalance of 2.566975502936061E-5 to a new imbalance of 6.580041563428285E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014213381090374027); MoveCostFunction : (multiplier=7.0, imbalance=0.0028888888888888888); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-11 08:53:14,684 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1220151015:83 , srv1333346751:83 , srv1435691219:83 , srv1447376454:83 , srv1574914465:83 , srv1858882404:83 , srv2037908504:83 , srv267542445:83 , srv541603207:83 , srv868346861:83 , srv1011394663:84 , srv1023843341:84 , srv1037286128:84 , srv1056530290:84 , srv1061952467:84 , srv1079592893:84 , srv1098996858:84 , srv1121979321:84 , srv1148442319:84 , srv1178814787:84 , srv1191463751:84 , srv1197742626:84 , srv1206698966:84 , srv1225550479:84 , srv1258598369:84 , srv1262790479:84 , srv1267299436:84 , srv1299609237:84 , srv1328339685:84 , srv1342921754:84 , srv1391070141:84 , srv145671912:84 , srv1475032090:84 , srv1510509599:84 , srv1521520822:84 , srv1535437603:84 , srv1559296291:84 , srv1561307979:84 , srv1652350375:84 , srv1657045099:84 , srv1675558784:84 , srv1676089203:84 , srv1681736855:84 , srv1738402719:84 , srv1740450080:84 , srv1769854735:84 , srv1770715970:84 , srv1794329322:84 , srv1825081754:84 , srv1878544030:84 , srv1948822516:84 , srv1977086210:84 , srv200607295:84 , srv20188671:84 , srv2058691131:84 , srv20672533:84 , srv2142092973:84 , srv255212112:84 , srv274059065:84 , srv383009719:84 , srv394578921:84 , srv42830804:84 , srv461457756:84 , srv463150279:84 , srv496791385:84 , srv502140118:84 , srv508712809:84 , srv529410407:84 , srv573788868:84 , srv589295711:84 , srv597489687:84 , srv610439811:84 , srv61114630:84 , srv619137708:84 , srv621711897:84 , srv636918746:84 , srv65072371:84 , srv654978708:84 , srv683975402:84 , srv697418086:84 , srv739025599:84 , srv771800711:84 , srv789971272:84 , srv831370006:84 , srv851300311:84 , srv900865760:84 , srv909896915:84 , srv962322707:84 , srv96645567:84 , srv1541976511:85 , srv1593832507:85 , srv1740416920:85 , srv1844374357:85 , srv356629517:85 , srv357377009:85 , srv399967985:85 , srv495054059:85 , srv775970449:85 , srv852554288:85 , srv1675632795:684 }
2024-02-11 08:53:14,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-11 08:53:14,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1574914465=33, srv1328339685=19, srv394578921=65, srv1225550479=14, srv61114630=80, srv1561307979=32, srv868346861=95, srv909896915=97, srv1023843341=1, srv1178814787=9, srv851300311=93, srv1740450080=43, srv357377009=63, srv1794329322=46, srv852554288=94, srv1475032090=26, srv789971272=91, srv145671912=25, srv1011394663=0, srv1197742626=11, srv200607295=53, srv1258598369=15, srv508712809=73, srv1220151015=13, srv597489687=78, srv1676089203=39, srv771800711=89, srv274059065=61, srv900865760=96, srv529410407=74, srv502140118=72, srv267542445=60, srv1521520822=28, srv654978708=85, srv1206698966=12, srv1037286128=2, srv42830804=67, srv20188671=54, srv255212112=59, srv541603207=75, srv1061952467=4, srv1447376454=24, srv636918746=83, srv2058691131=56, srv383009719=64, srv1299609237=18, srv1191463751=10, srv1391070141=22, srv1098996858=6, srv1333346751=20, srv1738402719=41, srv619137708=81, srv1510509599=27, srv1740416920=42, srv1262790479=16, srv1267299436=17, srv1825081754=47, srv1559296291=31, srv1535437603=29, srv1541976511=30, srv1770715970=45, srv697418086=87, srv1056530290=3, srv610439811=79, srv1681736855=40, srv1858882404=49, srv1079592893=5, srv1948822516=51, srv356629517=62, srv683975402=86, srv461457756=68, srv1657045099=36, srv1844374357=48, srv1435691219=23, srv1121979321=7, srv1977086210=52, srv96645567=99, srv775970449=90, srv20672533=57, srv495054059=70, srv1652350375=35, srv463150279=69, srv2142092973=58, srv1148442319=8, srv2037908504=55, srv589295711=77, srv1342921754=21, srv496791385=71, srv962322707=98, srv739025599=88, srv831370006=92, srv621711897=82, srv399967985=66, srv1593832507=34, srv1675632795=38, srv573788868=76, srv65072371=84, srv1675558784=37, srv1769854735=44, srv1878544030=50} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-11 08:53:14,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-11 08:53:14,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-11 08:53:14,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-11 08:53:14,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-11 08:53:14,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-11 08:53:14,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-11 08:53:14,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-11 08:53:14,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-11 08:53:14,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-11 08:53:14,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-11 08:53:14,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-11 08:53:14,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-11 08:53:14,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-11 08:53:14,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-11 08:53:14,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-11 08:53:14,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-11 08:53:14,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-11 08:53:14,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-11 08:53:14,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-11 08:53:14,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-11 08:53:14,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-11 08:53:14,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-11 08:53:14,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-11 08:53:14,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-11 08:53:14,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-11 08:53:14,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-11 08:53:14,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-11 08:53:14,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-11 08:53:14,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-11 08:53:14,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-11 08:53:14,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-02-11 08:53:14,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-02-11 08:53:14,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-02-11 08:53:14,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-02-11 08:53:14,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-02-11 08:53:14,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-02-11 08:53:14,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-02-11 08:53:14,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-02-11 08:53:14,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-02-11 08:53:14,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-02-11 08:53:14,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-02-11 08:53:14,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-02-11 08:53:14,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-02-11 08:53:14,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-02-11 08:53:14,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-02-11 08:53:14,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-02-11 08:53:14,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-02-11 08:53:14,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-02-11 08:53:14,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-02-11 08:53:14,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-02-11 08:53:14,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-02-11 08:53:14,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-02-11 08:53:14,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-02-11 08:53:14,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-02-11 08:53:14,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-02-11 08:53:14,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-02-11 08:53:14,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-02-11 08:53:14,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-02-11 08:53:14,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-02-11 08:53:14,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-02-11 08:53:14,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-02-11 08:53:14,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-02-11 08:53:14,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-02-11 08:53:14,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-02-11 08:53:14,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-02-11 08:53:14,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-02-11 08:53:14,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-02-11 08:53:14,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-02-11 08:53:14,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-02-11 08:53:14,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-02-11 08:53:14,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-02-11 08:53:14,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-02-11 08:53:14,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-02-11 08:53:14,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-02-11 08:53:14,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-02-11 08:53:14,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-02-11 08:53:14,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-02-11 08:53:14,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-02-11 08:53:14,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-02-11 08:53:14,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-02-11 08:53:14,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-02-11 08:53:14,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-02-11 08:53:14,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-02-11 08:53:14,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-02-11 08:53:14,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-02-11 08:53:14,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-02-11 08:53:14,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-02-11 08:53:14,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-02-11 08:53:14,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-02-11 08:53:14,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-02-11 08:53:14,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-02-11 08:53:14,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-02-11 08:53:14,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-02-11 08:53:14,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-02-11 08:53:14,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-02-11 08:53:14,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-02-11 08:53:14,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-02-11 08:53:14,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-02-11 08:53:14,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-02-11 08:53:14,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-02-11 08:53:14,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-02-11 08:53:14,805 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=6.397985672269699E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014213381090374027); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-11 08:53:14,805 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1220151015:83 , srv1333346751:83 , srv1435691219:83 , srv1447376454:83 , srv1574914465:83 , srv1858882404:83 , srv2037908504:83 , srv267542445:83 , srv541603207:83 , srv868346861:83 , srv1011394663:84 , srv1023843341:84 , srv1037286128:84 , srv1056530290:84 , srv1061952467:84 , srv1079592893:84 , srv1098996858:84 , srv1121979321:84 , srv1148442319:84 , srv1178814787:84 , srv1191463751:84 , srv1197742626:84 , srv1206698966:84 , srv1225550479:84 , srv1258598369:84 , srv1262790479:84 , srv1267299436:84 , srv1299609237:84 , srv1328339685:84 , srv1342921754:84 , srv1391070141:84 , srv145671912:84 , srv1475032090:84 , srv1510509599:84 , srv1521520822:84 , srv1535437603:84 , srv1559296291:84 , srv1561307979:84 , srv1652350375:84 , srv1657045099:84 , srv1675558784:84 , srv1676089203:84 , srv1681736855:84 , srv1738402719:84 , srv1740450080:84 , srv1769854735:84 , srv1770715970:84 , srv1794329322:84 , srv1825081754:84 , srv1878544030:84 , srv1948822516:84 , srv1977086210:84 , srv200607295:84 , srv20188671:84 , srv2058691131:84 , srv20672533:84 , srv2142092973:84 , srv255212112:84 , srv274059065:84 , srv383009719:84 , srv394578921:84 , srv42830804:84 , srv461457756:84 , srv463150279:84 , srv496791385:84 , srv502140118:84 , srv508712809:84 , srv529410407:84 , srv573788868:84 , srv589295711:84 , srv597489687:84 , srv610439811:84 , srv61114630:84 , srv619137708:84 , srv621711897:84 , srv636918746:84 , srv65072371:84 , srv654978708:84 , srv683975402:84 , srv697418086:84 , srv739025599:84 , srv771800711:84 , srv789971272:84 , srv831370006:84 , srv851300311:84 , srv900865760:84 , srv909896915:84 , srv962322707:84 , srv96645567:84 , srv1541976511:85 , srv1593832507:85 , srv1740416920:85 , srv1844374357:85 , srv356629517:85 , srv357377009:85 , srv399967985:85 , srv495054059:85 , srv775970449:85 , srv852554288:85 , srv1675632795:684 }
2024-02-11 08:53:14,828 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=326 (was 276) - SystemLoadAverage LEAK? -, ProcessCount=167 (was 167), AvailableMemoryMB=2451 (was 3201)
2024-02-11 08:53:14,835 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=326, ProcessCount=167, AvailableMemoryMB=2451
2024-02-11 08:53:14,835 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-02-11 08:53:14,836 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2024-02-11 08:53:14,845 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1025160908:84 , srv1050900935:84 , srv105189666:84 , srv108821752:84 , srv1097871696:84 , srv1156272930:84 , srv1234164494:84 , srv1265896510:84 , srv152100534:84 , srv1572384715:84 , srv1619607105:84 , srv1680913814:84 , srv1914502467:84 , srv1930309662:84 , srv2019431605:84 , srv2071753220:84 , srv22729835:84 , srv246604899:84 , srv307802924:84 , srv443191690:84 , srv546147374:84 , srv560430897:84 , srv641393987:84 , srv646346666:84 , srv666528215:84 , srv727944274:84 , srv875582246:84 , srv893346634:84 , srv951335725:84 , srv99794596:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2024-02-11 08:53:14,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-11 08:53:14,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv727944274=25, srv1156272930=5, srv1930309662=13, srv560430897=21, srv893346634=27, srv108821752=3, srv2019431605=14, srv666528215=24, srv2071753220=15, srv1914502467=12, srv1025160908=0, srv443191690=19, srv641393987=22, srv951335725=28, srv1234164494=6, srv1572384715=9, srv105189666=2, srv1680913814=11, srv99794596=29, srv1265896510=7, srv22729835=16, srv246604899=17, srv546147374=20, srv1097871696=4, srv152100534=8, srv646346666=23, srv875582246=26, srv1050900935=1, srv1619607105=10, srv307802924=18} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-11 08:53:14,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-11 08:53:14,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-11 08:53:14,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-11 08:53:14,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-11 08:53:14,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-11 08:53:14,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-11 08:53:14,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-11 08:53:14,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-11 08:53:14,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-11 08:53:14,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-11 08:53:14,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-11 08:53:14,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-11 08:53:14,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-11 08:53:14,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-11 08:53:14,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-11 08:53:14,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-11 08:53:14,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-11 08:53:14,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-11 08:53:14,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-11 08:53:14,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-11 08:53:14,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-11 08:53:14,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-11 08:53:14,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-11 08:53:14,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-11 08:53:14,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-11 08:53:14,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-11 08:53:14,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-11 08:53:14,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-11 08:53:14,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-11 08:53:14,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-11 08:53:14,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-02-11 08:53:14,870 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.064116614395483, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-11 08:53:14,870 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912395072622642, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.064116614395483, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2024-02-11 08:55:14,858 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6336454 different iterations.  Found a solution that moves 1845 regions; Going from a computed imbalance of 0.9912395072622642 to a new imbalance of 4.306322040866546E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6833333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-11 08:55:14,865 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1025160908:90 , srv1050900935:90 , srv105189666:90 , srv108821752:90 , srv1097871696:90 , srv1156272930:90 , srv1234164494:90 , srv1265896510:90 , srv152100534:90 , srv1572384715:90 , srv1619607105:90 , srv1680913814:90 , srv1914502467:90 , srv1930309662:90 , srv2019431605:90 , srv2071753220:90 , srv22729835:90 , srv246604899:90 , srv307802924:90 , srv443191690:90 , srv546147374:90 , srv560430897:90 , srv641393987:90 , srv646346666:90 , srv666528215:90 , srv727944274:90 , srv875582246:90 , srv893346634:90 , srv951335725:90 , srv99794596:90 }
2024-02-11 08:55:14,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-11 08:55:14,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv727944274=25, srv1156272930=5, srv1930309662=13, srv560430897=21, srv893346634=27, srv108821752=3, srv2019431605=14, srv666528215=24, srv2071753220=15, srv1914502467=12, srv1025160908=0, srv443191690=19, srv641393987=22, srv951335725=28, srv1234164494=6, srv1572384715=9, srv105189666=2, srv1680913814=11, srv99794596=29, srv1265896510=7, srv22729835=16, srv246604899=17, srv546147374=20, srv1097871696=4, srv152100534=8, srv646346666=23, srv875582246=26, srv1050900935=1, srv1619607105=10, srv307802924=18} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-11 08:55:14,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-11 08:55:14,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-11 08:55:14,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-11 08:55:14,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-11 08:55:14,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-11 08:55:14,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-11 08:55:14,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-11 08:55:14,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-11 08:55:14,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-11 08:55:14,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-11 08:55:14,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-11 08:55:14,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-11 08:55:14,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-11 08:55:14,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-11 08:55:14,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-11 08:55:14,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-11 08:55:14,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-11 08:55:14,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-11 08:55:14,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-11 08:55:14,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-11 08:55:14,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-11 08:55:14,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-11 08:55:14,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-11 08:55:14,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-11 08:55:14,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-11 08:55:14,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-11 08:55:14,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-11 08:55:14,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-11 08:55:14,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-11 08:55:14,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-11 08:55:14,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-02-11 08:55:14,885 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-11 08:55:14,885 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1025160908:90 , srv1050900935:90 , srv105189666:90 , srv108821752:90 , srv1097871696:90 , srv1156272930:90 , srv1234164494:90 , srv1265896510:90 , srv152100534:90 , srv1572384715:90 , srv1619607105:90 , srv1680913814:90 , srv1914502467:90 , srv1930309662:90 , srv2019431605:90 , srv2071753220:90 , srv22729835:90 , srv246604899:90 , srv307802924:90 , srv443191690:90 , srv546147374:90 , srv560430897:90 , srv641393987:90 , srv646346666:90 , srv666528215:90 , srv727944274:90 , srv875582246:90 , srv893346634:90 , srv951335725:90 , srv99794596:90 }
2024-02-11 08:55:14,897 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=219 (was 326), ProcessCount=167 (was 167), AvailableMemoryMB=1993 (was 2451)