2024-03-20 16:54:39,083 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2024-03-20 16:54:39,421 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2024-03-20 16:54:39,527 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2024-03-20 16:54:39,592 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-03-20 16:54:39,599 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-03-20 16:54:39,616 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=610, ProcessCount=193, AvailableMemoryMB=3007
2024-03-20 16:54:39,617 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-03-20 16:54:39,618 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-03-20 16:54:39,768 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1010636852:84 , srv1075228465:84 , srv1082070072:84 , srv1126701451:84 , srv1140845359:84 , srv1157103363:84 , srv1161962706:84 , srv1179796924:84 , srv124529410:84 , srv1263085573:84 , srv1318334870:84 , srv1329731745:84 , srv1332938545:84 , srv1337952446:84 , srv1381080443:84 , srv1381973416:84 , srv1395594923:84 , srv1418280300:84 , srv1482958952:84 , srv1493039153:84 , srv1518465775:84 , srv1524117808:84 , srv1530397171:84 , srv1540884956:84 , srv158086359:84 , srv1582204142:84 , srv1642334392:84 , srv1648308996:84 , srv1688566269:84 , srv1715035624:84 , srv1721390741:84 , srv1729135098:84 , srv1735503112:84 , srv1739560364:84 , srv1744117989:84 , srv1769212565:84 , srv1780020215:84 , srv1788202218:84 , srv1799731772:84 , srv1805255051:84 , srv1816034785:84 , srv1825434873:84 , srv1830098496:84 , srv1833475319:84 , srv1838249150:84 , srv1911404740:84 , srv1927614527:84 , srv1946892778:84 , srv195157664:84 , srv1984687355:84 , srv1994000141:84 , srv2037512389:84 , srv2043823485:84 , srv2049238365:84 , srv205030921:84 , srv2060581181:84 , srv2069596160:84 , srv2080756296:84 , srv208278686:84 , srv2087294807:84 , srv2116471643:84 , srv2134022843:84 , srv2138366135:84 , srv223495324:84 , srv270629074:84 , srv305085847:84 , srv380459719:84 , srv387160648:84 , srv390390511:84 , srv395131930:84 , srv402923580:84 , srv444338984:84 , srv450057666:84 , srv459588057:84 , srv463910070:84 , srv468232300:84 , srv485296249:84 , srv514014712:84 , srv564227331:84 , srv58538325:84 , srv606020100:84 , srv612351158:84 , srv628626359:84 , srv649323056:84 , srv659851472:84 , srv683924424:84 , srv684755254:84 , srv709481546:84 , srv718604911:84 , srv727478809:84 , srv728826233:84 , srv754929532:84 , srv76969324:84 , srv786210135:84 , srv8400371:84 , srv846821884:84 , srv853380171:84 , srv889065435:84 , srv927909894:84 , srv404791997:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2024-03-20 16:54:39,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-03-20 16:54:39,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1493039153=19, srv709481546=88, srv1830098496=42, srv158086359=24, srv387160648=67, srv1946892778=47, srv728826233=91, srv1642334392=26, srv1395594923=16, srv1838249150=44, srv1161962706=6, srv2116471643=60, srv683924424=86, srv1715035624=29, srv606020100=81, srv659851472=85, srv1524117808=21, srv1518465775=20, srv1648308996=27, srv58538325=80, srv684755254=87, srv754929532=92, srv1582204142=25, srv195157664=48, srv727478809=90, srv2060581181=55, srv1381973416=15, srv2049238365=53, srv404791997=71, srv76969324=93, srv270629074=64, srv1994000141=50, srv649323056=84, srv2087294807=59, srv1126701451=3, srv468232300=76, srv2043823485=52, srv514014712=78, srv889065435=98, srv1140845359=4, srv205030921=54, srv1381080443=14, srv1729135098=31, srv2134022843=61, srv1911404740=45, srv2037512389=51, srv444338984=72, srv1744117989=34, srv1825434873=41, srv1329731745=11, srv1530397171=22, srv1805255051=39, srv2138366135=62, srv1418280300=17, srv208278686=58, srv485296249=77, srv628626359=83, srv927909894=99, srv1318334870=10, srv1735503112=32, srv390390511=68, srv1799731772=38, srv380459719=66, srv1482958952=18, srv459588057=74, srv853380171=97, srv1263085573=9, srv1984687355=49, srv402923580=70, srv786210135=94, srv1179796924=7, srv1739560364=33, srv1927614527=46, srv718604911=89, srv1780020215=36, srv223495324=63, srv846821884=96, srv612351158=82, srv395131930=69, srv2069596160=56, srv1337952446=13, srv1082070072=2, srv450057666=73, srv1332938545=12, srv1010636852=0, srv564227331=79, srv1157103363=5, srv124529410=8, srv1075228465=1, srv1788202218=37, srv305085847=65, srv463910070=75, srv1769212565=35, srv1816034785=40, srv8400371=95, srv1721390741=30, srv1833475319=43, srv1540884956=23, srv1688566269=28, srv2080756296=57} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-03-20 16:54:39,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-03-20 16:54:39,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-03-20 16:54:39,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-03-20 16:54:39,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-03-20 16:54:39,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-03-20 16:54:39,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-03-20 16:54:39,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-03-20 16:54:39,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-03-20 16:54:39,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-03-20 16:54:39,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-03-20 16:54:39,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-03-20 16:54:39,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-03-20 16:54:39,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-03-20 16:54:39,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-03-20 16:54:39,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-03-20 16:54:39,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-03-20 16:54:39,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-03-20 16:54:39,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-03-20 16:54:39,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-03-20 16:54:39,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-03-20 16:54:39,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-03-20 16:54:39,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-03-20 16:54:39,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-03-20 16:54:39,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-03-20 16:54:39,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-03-20 16:54:39,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-03-20 16:54:39,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-03-20 16:54:39,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-03-20 16:54:39,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-03-20 16:54:39,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-03-20 16:54:39,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-03-20 16:54:39,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-03-20 16:54:39,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-03-20 16:54:39,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-03-20 16:54:39,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-03-20 16:54:39,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-03-20 16:54:39,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-03-20 16:54:39,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-03-20 16:54:39,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-03-20 16:54:39,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-03-20 16:54:39,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-03-20 16:54:39,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-03-20 16:54:39,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-03-20 16:54:39,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-03-20 16:54:39,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-03-20 16:54:39,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-03-20 16:54:39,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-03-20 16:54:39,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-03-20 16:54:39,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-03-20 16:54:39,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-03-20 16:54:39,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-03-20 16:54:39,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-03-20 16:54:39,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-03-20 16:54:39,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-03-20 16:54:39,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-03-20 16:54:39,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-03-20 16:54:39,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-03-20 16:54:39,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-03-20 16:54:39,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-03-20 16:54:39,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-03-20 16:54:39,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-03-20 16:54:39,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-03-20 16:54:39,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-03-20 16:54:39,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-03-20 16:54:39,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-03-20 16:54:39,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-03-20 16:54:39,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-03-20 16:54:39,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-03-20 16:54:39,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-03-20 16:54:39,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-03-20 16:54:39,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-03-20 16:54:39,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-03-20 16:54:39,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-03-20 16:54:39,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-03-20 16:54:39,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-03-20 16:54:39,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-03-20 16:54:39,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-03-20 16:54:39,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-03-20 16:54:39,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-03-20 16:54:39,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-03-20 16:54:39,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-03-20 16:54:39,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-03-20 16:54:39,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-03-20 16:54:39,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-03-20 16:54:39,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-03-20 16:54:39,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-03-20 16:54:39,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-03-20 16:54:39,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-03-20 16:54:39,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-03-20 16:54:39,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-03-20 16:54:39,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-03-20 16:54:39,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-03-20 16:54:39,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-03-20 16:54:39,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-03-20 16:54:39,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-03-20 16:54:39,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-03-20 16:54:39,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-03-20 16:54:39,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-03-20 16:54:39,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-03-20 16:54:39,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-03-20 16:54:39,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-03-20 16:54:40,322 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-20 16:54:40,323 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-03-20 16:54:40,323 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-03-20 16:54:40,872 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 953 ms to try 5000 different iterations.  Found a solution that moves 4691 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17666173970781904. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04212345942162268, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05874291035858016, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5212222222222223, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04212345942162268, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17208333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23591666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-20 16:54:40,884 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1381080443:80 , srv1530397171:80 , srv1075228465:81 , srv1994000141:81 , srv1715035624:82 , srv2037512389:82 , srv459588057:82 , srv485296249:82 , srv606020100:82 , srv612351158:82 , srv718604911:82 , srv1082070072:83 , srv1126701451:83 , srv1332938545:83 , srv1582204142:83 , srv1735503112:83 , srv1927614527:83 , srv2138366135:83 , srv463910070:83 , srv659851472:83 , srv1161962706:84 , srv1337952446:84 , srv1395594923:84 , srv1482958952:84 , srv1984687355:84 , srv205030921:84 , srv709481546:84 , srv728826233:84 , srv889065435:84 , srv1157103363:85 , srv1493039153:85 , srv1739560364:85 , srv1744117989:85 , srv1788202218:85 , srv1830098496:85 , srv1911404740:85 , srv2043823485:85 , srv208278686:85 , srv402923580:85 , srv514014712:85 , srv628626359:85 , srv649323056:85 , srv727478809:85 , srv1318334870:86 , srv1642334392:86 , srv1648308996:86 , srv1721390741:86 , srv1769212565:86 , srv1825434873:86 , srv1946892778:86 , srv2049238365:86 , srv2080756296:86 , srv305085847:86 , srv395131930:86 , srv444338984:86 , srv468232300:86 , srv8400371:86 , srv853380171:86 , srv124529410:87 , srv1329731745:87 , srv158086359:87 , srv1816034785:87 , srv2060581181:87 , srv387160648:87 , srv754929532:87 , srv76969324:87 , srv846821884:87 , srv1140845359:88 , srv1688566269:88 , srv1838249150:88 , srv2069596160:88 , srv270629074:88 , srv450057666:88 , srv684755254:88 , srv786210135:88 , srv1179796924:89 , srv1518465775:89 , srv1540884956:89 , srv1729135098:89 , srv1780020215:89 , srv1799731772:89 , srv1805255051:89 , srv195157664:89 , srv2087294807:89 , srv2116471643:89 , srv380459719:89 , srv564227331:89 , srv927909894:89 , srv1010636852:90 , srv1263085573:90 , srv1524117808:90 , srv2134022843:90 , srv58538325:90 , srv223495324:92 , srv390390511:92 , srv683924424:92 , srv1418280300:93 , srv1833475319:96 , srv1381973416:98 , srv404791997:464 }
2024-03-20 16:54:40,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-03-20 16:54:40,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1493039153=19, srv709481546=88, srv1830098496=42, srv158086359=24, srv387160648=67, srv1946892778=47, srv728826233=91, srv1642334392=26, srv1395594923=16, srv1838249150=44, srv1161962706=6, srv2116471643=60, srv683924424=86, srv1715035624=29, srv606020100=81, srv659851472=85, srv1524117808=21, srv1518465775=20, srv1648308996=27, srv58538325=80, srv684755254=87, srv754929532=92, srv1582204142=25, srv195157664=48, srv727478809=90, srv2060581181=55, srv1381973416=15, srv2049238365=53, srv404791997=71, srv76969324=93, srv270629074=64, srv1994000141=50, srv649323056=84, srv2087294807=59, srv1126701451=3, srv468232300=76, srv2043823485=52, srv514014712=78, srv889065435=98, srv1140845359=4, srv205030921=54, srv1381080443=14, srv1729135098=31, srv2134022843=61, srv1911404740=45, srv2037512389=51, srv444338984=72, srv1744117989=34, srv1825434873=41, srv1329731745=11, srv1530397171=22, srv1805255051=39, srv2138366135=62, srv1418280300=17, srv208278686=58, srv485296249=77, srv628626359=83, srv927909894=99, srv1318334870=10, srv1735503112=32, srv390390511=68, srv1799731772=38, srv380459719=66, srv1482958952=18, srv459588057=74, srv853380171=97, srv1263085573=9, srv1984687355=49, srv402923580=70, srv786210135=94, srv1179796924=7, srv1739560364=33, srv1927614527=46, srv718604911=89, srv1780020215=36, srv223495324=63, srv846821884=96, srv612351158=82, srv395131930=69, srv2069596160=56, srv1337952446=13, srv1082070072=2, srv450057666=73, srv1332938545=12, srv1010636852=0, srv564227331=79, srv1157103363=5, srv124529410=8, srv1075228465=1, srv1788202218=37, srv305085847=65, srv463910070=75, srv1769212565=35, srv1816034785=40, srv8400371=95, srv1721390741=30, srv1833475319=43, srv1540884956=23, srv1688566269=28, srv2080756296=57} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-03-20 16:54:40,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-03-20 16:54:40,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-03-20 16:54:40,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-03-20 16:54:40,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-03-20 16:54:40,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-03-20 16:54:40,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-03-20 16:54:40,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-03-20 16:54:40,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-03-20 16:54:40,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-03-20 16:54:40,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-03-20 16:54:40,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-03-20 16:54:40,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-03-20 16:54:40,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-03-20 16:54:40,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-03-20 16:54:40,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-03-20 16:54:40,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-03-20 16:54:40,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-03-20 16:54:40,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-03-20 16:54:40,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-03-20 16:54:40,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-03-20 16:54:40,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-03-20 16:54:40,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-03-20 16:54:40,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-03-20 16:54:40,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-03-20 16:54:40,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-03-20 16:54:40,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-03-20 16:54:40,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-03-20 16:54:40,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-03-20 16:54:40,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-03-20 16:54:40,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-03-20 16:54:40,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-03-20 16:54:40,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-03-20 16:54:40,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-03-20 16:54:40,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-03-20 16:54:40,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-03-20 16:54:40,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-03-20 16:54:40,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-03-20 16:54:40,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-03-20 16:54:40,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-03-20 16:54:40,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-03-20 16:54:40,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-03-20 16:54:40,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-03-20 16:54:40,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-03-20 16:54:40,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-03-20 16:54:40,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-03-20 16:54:40,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-03-20 16:54:40,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-03-20 16:54:40,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-03-20 16:54:40,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-03-20 16:54:40,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-03-20 16:54:40,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-03-20 16:54:40,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-03-20 16:54:40,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-03-20 16:54:40,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-03-20 16:54:40,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-03-20 16:54:40,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-03-20 16:54:40,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-03-20 16:54:40,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-03-20 16:54:40,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-03-20 16:54:40,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-03-20 16:54:40,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-03-20 16:54:40,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-03-20 16:54:40,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-03-20 16:54:40,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-03-20 16:54:40,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-03-20 16:54:40,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-03-20 16:54:40,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-03-20 16:54:40,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-03-20 16:54:40,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-03-20 16:54:40,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-03-20 16:54:40,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-03-20 16:54:40,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-03-20 16:54:40,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-03-20 16:54:40,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-03-20 16:54:40,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-03-20 16:54:40,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-03-20 16:54:40,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-03-20 16:54:40,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-03-20 16:54:40,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-03-20 16:54:40,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-03-20 16:54:40,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-03-20 16:54:40,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-03-20 16:54:40,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-03-20 16:54:40,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-03-20 16:54:40,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-03-20 16:54:40,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-03-20 16:54:40,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-03-20 16:54:40,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-03-20 16:54:40,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-03-20 16:54:40,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-03-20 16:54:40,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-03-20 16:54:40,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-03-20 16:54:40,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-03-20 16:54:40,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-03-20 16:54:40,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-03-20 16:54:40,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-03-20 16:54:40,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-03-20 16:54:40,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-03-20 16:54:40,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-03-20 16:54:40,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-03-20 16:54:40,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-03-20 16:54:41,107 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04212345942162268, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05874291035858016, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04212345942162268, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17208333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23591666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-20 16:54:41,107 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-03-20 16:54:41,108 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17662889262376422, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04212345942162268, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05874291035858016, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04212345942162268, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17208333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23591666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-03-20 16:54:41,353 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 384 ms to try 5000 different iterations.  Found a solution that moves 2625 regions; Going from a computed imbalance of 0.17662889262376422 to a new imbalance of 0.01422000474464937. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02486827750043483); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03500521045070422, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2916666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02486827750043483); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011166666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-20 16:54:41,366 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1381973416:75 , srv1833475319:76 , srv1010636852:79 , srv1418280300:79 , srv683924424:79 , srv1263085573:80 , srv390390511:80 , srv58538325:80 , srv927909894:80 , srv2087294807:81 , srv223495324:81 , srv1518465775:82 , srv1524117808:82 , srv1805255051:82 , srv1838249150:82 , srv2069596160:82 , srv2134022843:82 , srv754929532:82 , srv1179796924:83 , srv124529410:83 , srv195157664:83 , srv450057666:83 , srv684755254:83 , srv1318334870:84 , srv1540884956:84 , srv1780020215:84 , srv1799731772:84 , srv1946892778:84 , srv2060581181:84 , srv2116471643:84 , srv270629074:84 , srv380459719:84 , srv387160648:84 , srv402923580:84 , srv786210135:84 , srv8400371:84 , srv1337952446:85 , srv1493039153:85 , srv1721390741:85 , srv1729135098:85 , srv1739560364:85 , srv1744117989:85 , srv1769212565:85 , srv1816034785:85 , srv1329731745:86 , srv1332938545:86 , srv1642334392:86 , srv1825434873:86 , srv2049238365:86 , srv395131930:86 , srv468232300:86 , srv564227331:86 , srv628626359:86 , srv659851472:86 , srv727478809:86 , srv76969324:86 , srv846821884:86 , srv889065435:86 , srv1082070072:87 , srv1126701451:87 , srv1140845359:87 , srv1161962706:87 , srv1395594923:87 , srv1482958952:87 , srv158086359:87 , srv1582204142:87 , srv1648308996:87 , srv1688566269:87 , srv1788202218:87 , srv2080756296:87 , srv2138366135:87 , srv649323056:87 , srv709481546:87 , srv728826233:87 , srv853380171:87 , srv1830098496:88 , srv1984687355:88 , srv208278686:88 , srv305085847:88 , srv514014712:88 , srv1157103363:89 , srv1530397171:89 , srv1911404740:89 , srv1927614527:89 , srv2043823485:89 , srv205030921:89 , srv1994000141:90 , srv444338984:90 , srv463910070:90 , srv606020100:90 , srv612351158:90 , srv1075228465:91 , srv1381080443:91 , srv1715035624:91 , srv1735503112:91 , srv2037512389:91 , srv718604911:91 , srv485296249:92 , srv459588057:93 , srv404791997:531 }
2024-03-20 16:54:41,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-03-20 16:54:41,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1493039153=19, srv709481546=88, srv1830098496=42, srv158086359=24, srv387160648=67, srv1946892778=47, srv728826233=91, srv1642334392=26, srv1395594923=16, srv1838249150=44, srv1161962706=6, srv2116471643=60, srv683924424=86, srv1715035624=29, srv606020100=81, srv659851472=85, srv1524117808=21, srv1518465775=20, srv1648308996=27, srv58538325=80, srv684755254=87, srv754929532=92, srv1582204142=25, srv195157664=48, srv727478809=90, srv2060581181=55, srv1381973416=15, srv2049238365=53, srv404791997=71, srv76969324=93, srv270629074=64, srv1994000141=50, srv649323056=84, srv2087294807=59, srv1126701451=3, srv468232300=76, srv2043823485=52, srv514014712=78, srv889065435=98, srv1140845359=4, srv205030921=54, srv1381080443=14, srv1729135098=31, srv2134022843=61, srv1911404740=45, srv2037512389=51, srv444338984=72, srv1744117989=34, srv1825434873=41, srv1329731745=11, srv1530397171=22, srv1805255051=39, srv2138366135=62, srv1418280300=17, srv208278686=58, srv485296249=77, srv628626359=83, srv927909894=99, srv1318334870=10, srv1735503112=32, srv390390511=68, srv1799731772=38, srv380459719=66, srv1482958952=18, srv459588057=74, srv853380171=97, srv1263085573=9, srv1984687355=49, srv402923580=70, srv786210135=94, srv1179796924=7, srv1739560364=33, srv1927614527=46, srv718604911=89, srv1780020215=36, srv223495324=63, srv846821884=96, srv612351158=82, srv395131930=69, srv2069596160=56, srv1337952446=13, srv1082070072=2, srv450057666=73, srv1332938545=12, srv1010636852=0, srv564227331=79, srv1157103363=5, srv124529410=8, srv1075228465=1, srv1788202218=37, srv305085847=65, srv463910070=75, srv1769212565=35, srv1816034785=40, srv8400371=95, srv1721390741=30, srv1833475319=43, srv1540884956=23, srv1688566269=28, srv2080756296=57} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-03-20 16:54:41,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-03-20 16:54:41,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-03-20 16:54:41,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-03-20 16:54:41,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-03-20 16:54:41,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-03-20 16:54:41,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-03-20 16:54:41,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-03-20 16:54:41,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-03-20 16:54:41,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-03-20 16:54:41,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-03-20 16:54:41,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-03-20 16:54:41,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-03-20 16:54:41,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-03-20 16:54:41,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-03-20 16:54:41,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-03-20 16:54:41,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-03-20 16:54:41,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-03-20 16:54:41,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-03-20 16:54:41,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-03-20 16:54:41,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-03-20 16:54:41,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-03-20 16:54:41,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-03-20 16:54:41,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-03-20 16:54:41,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-03-20 16:54:41,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-03-20 16:54:41,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-03-20 16:54:41,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-03-20 16:54:41,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-03-20 16:54:41,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-03-20 16:54:41,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-03-20 16:54:41,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-03-20 16:54:41,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-03-20 16:54:41,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-03-20 16:54:41,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-03-20 16:54:41,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-03-20 16:54:41,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-03-20 16:54:41,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-03-20 16:54:41,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-03-20 16:54:41,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-03-20 16:54:41,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-03-20 16:54:41,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-03-20 16:54:41,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-03-20 16:54:41,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-03-20 16:54:41,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-03-20 16:54:41,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-03-20 16:54:41,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-03-20 16:54:41,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-03-20 16:54:41,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-03-20 16:54:41,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-03-20 16:54:41,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-03-20 16:54:41,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-03-20 16:54:41,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-03-20 16:54:41,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-03-20 16:54:41,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-03-20 16:54:41,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-03-20 16:54:41,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-03-20 16:54:41,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-03-20 16:54:41,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-03-20 16:54:41,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-03-20 16:54:41,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-03-20 16:54:41,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-03-20 16:54:41,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-03-20 16:54:41,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-03-20 16:54:41,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-03-20 16:54:41,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-03-20 16:54:41,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-03-20 16:54:41,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-03-20 16:54:41,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-03-20 16:54:41,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-03-20 16:54:41,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-03-20 16:54:41,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-03-20 16:54:41,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-03-20 16:54:41,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-03-20 16:54:41,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-03-20 16:54:41,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-03-20 16:54:41,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-03-20 16:54:41,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-03-20 16:54:41,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-03-20 16:54:41,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-03-20 16:54:41,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-03-20 16:54:41,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-03-20 16:54:41,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-03-20 16:54:41,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-03-20 16:54:41,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-03-20 16:54:41,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-03-20 16:54:41,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-03-20 16:54:41,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-03-20 16:54:41,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-03-20 16:54:41,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-03-20 16:54:41,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-03-20 16:54:41,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-03-20 16:54:41,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-03-20 16:54:41,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-03-20 16:54:41,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-03-20 16:54:41,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-03-20 16:54:41,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-03-20 16:54:41,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-03-20 16:54:41,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-03-20 16:54:41,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-03-20 16:54:41,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-03-20 16:54:41,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-03-20 16:54:41,546 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02486827750043483); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03500521045070422, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02486827750043483); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011166666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-20 16:54:41,546 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-03-20 16:54:41,547 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.014201624101792014, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02486827750043483); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03500521045070422, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02486827750043483); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011166666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-03-20 16:54:41,905 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 483 ms to try 5000 different iterations.  Found a solution that moves 1008 regions; Going from a computed imbalance of 0.014201624101792014 to a new imbalance of 7.887153639131254E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0077881924820476555); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.010314906998965449); MoveCostFunction : (multiplier=7.0, imbalance=0.112, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0077881924820476555); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-20 16:54:41,907 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1140845359:81 , srv1688566269:82 , srv1729135098:82 , srv444338984:82 , srv459588057:82 , srv564227331:82 , srv1381973416:83 , srv2043823485:83 , srv485296249:83 , srv1157103363:84 , srv1540884956:84 , srv158086359:84 , srv1648308996:84 , srv1780020215:84 , srv1799731772:84 , srv1830098496:84 , srv1911404740:84 , srv1927614527:84 , srv205030921:84 , srv223495324:84 , srv305085847:84 , srv612351158:84 , srv76969324:84 , srv1075228465:85 , srv1179796924:85 , srv1329731745:85 , srv1418280300:85 , srv1735503112:85 , srv1788202218:85 , srv1833475319:85 , srv1984687355:85 , srv2037512389:85 , srv2080756296:85 , srv208278686:85 , srv2116471643:85 , srv270629074:85 , srv380459719:85 , srv395131930:85 , srv463910070:85 , srv514014712:85 , srv606020100:85 , srv649323056:85 , srv718604911:85 , srv727478809:85 , srv728826233:85 , srv786210135:85 , srv846821884:85 , srv853380171:85 , srv124529410:86 , srv1381080443:86 , srv1395594923:86 , srv1518465775:86 , srv1715035624:86 , srv1721390741:86 , srv1769212565:86 , srv1805255051:86 , srv1816034785:86 , srv1825434873:86 , srv195157664:86 , srv1994000141:86 , srv2049238365:86 , srv2060581181:86 , srv2069596160:86 , srv2134022843:86 , srv387160648:86 , srv390390511:86 , srv468232300:86 , srv628626359:86 , srv684755254:86 , srv709481546:86 , srv1082070072:87 , srv1126701451:87 , srv1263085573:87 , srv1318334870:87 , srv1337952446:87 , srv1482958952:87 , srv1524117808:87 , srv1582204142:87 , srv1642334392:87 , srv1744117989:87 , srv1946892778:87 , srv2087294807:87 , srv450057666:87 , srv58538325:87 , srv754929532:87 , srv8400371:87 , srv889065435:87 , srv1010636852:88 , srv1161962706:88 , srv1332938545:88 , srv1493039153:88 , srv1530397171:88 , srv1739560364:88 , srv1838249150:88 , srv2138366135:88 , srv402923580:88 , srv659851472:88 , srv683924424:88 , srv927909894:88 , srv404791997:532 }
2024-03-20 16:54:41,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-03-20 16:54:41,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1493039153=19, srv709481546=88, srv1830098496=42, srv158086359=24, srv387160648=67, srv1946892778=47, srv728826233=91, srv1642334392=26, srv1395594923=16, srv1838249150=44, srv1161962706=6, srv2116471643=60, srv683924424=86, srv1715035624=29, srv606020100=81, srv659851472=85, srv1524117808=21, srv1518465775=20, srv1648308996=27, srv58538325=80, srv684755254=87, srv754929532=92, srv1582204142=25, srv195157664=48, srv727478809=90, srv2060581181=55, srv1381973416=15, srv2049238365=53, srv404791997=71, srv76969324=93, srv270629074=64, srv1994000141=50, srv649323056=84, srv2087294807=59, srv1126701451=3, srv468232300=76, srv2043823485=52, srv514014712=78, srv889065435=98, srv1140845359=4, srv205030921=54, srv1381080443=14, srv1729135098=31, srv2134022843=61, srv1911404740=45, srv2037512389=51, srv444338984=72, srv1744117989=34, srv1825434873=41, srv1329731745=11, srv1530397171=22, srv1805255051=39, srv2138366135=62, srv1418280300=17, srv208278686=58, srv485296249=77, srv628626359=83, srv927909894=99, srv1318334870=10, srv1735503112=32, srv390390511=68, srv1799731772=38, srv380459719=66, srv1482958952=18, srv459588057=74, srv853380171=97, srv1263085573=9, srv1984687355=49, srv402923580=70, srv786210135=94, srv1179796924=7, srv1739560364=33, srv1927614527=46, srv718604911=89, srv1780020215=36, srv223495324=63, srv846821884=96, srv612351158=82, srv395131930=69, srv2069596160=56, srv1337952446=13, srv1082070072=2, srv450057666=73, srv1332938545=12, srv1010636852=0, srv564227331=79, srv1157103363=5, srv124529410=8, srv1075228465=1, srv1788202218=37, srv305085847=65, srv463910070=75, srv1769212565=35, srv1816034785=40, srv8400371=95, srv1721390741=30, srv1833475319=43, srv1540884956=23, srv1688566269=28, srv2080756296=57} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-03-20 16:54:41,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-03-20 16:54:41,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-03-20 16:54:41,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-03-20 16:54:41,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-03-20 16:54:41,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-03-20 16:54:41,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-03-20 16:54:41,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-03-20 16:54:41,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-03-20 16:54:41,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-03-20 16:54:41,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-03-20 16:54:41,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-03-20 16:54:41,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-03-20 16:54:41,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-03-20 16:54:41,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-03-20 16:54:41,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-03-20 16:54:41,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-03-20 16:54:41,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-03-20 16:54:41,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-03-20 16:54:41,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-03-20 16:54:41,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-03-20 16:54:41,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-03-20 16:54:41,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-03-20 16:54:41,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-03-20 16:54:41,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-03-20 16:54:41,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-03-20 16:54:41,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-03-20 16:54:41,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-03-20 16:54:41,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-03-20 16:54:41,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-03-20 16:54:41,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-03-20 16:54:41,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-03-20 16:54:41,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-03-20 16:54:41,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-03-20 16:54:41,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-03-20 16:54:41,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-03-20 16:54:41,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-03-20 16:54:41,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-03-20 16:54:41,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-03-20 16:54:41,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-03-20 16:54:41,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-03-20 16:54:41,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-03-20 16:54:41,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-03-20 16:54:41,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-03-20 16:54:41,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-03-20 16:54:41,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-03-20 16:54:41,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-03-20 16:54:41,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-03-20 16:54:41,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-03-20 16:54:41,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-03-20 16:54:41,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-03-20 16:54:41,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-03-20 16:54:41,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-03-20 16:54:41,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-03-20 16:54:41,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-03-20 16:54:41,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-03-20 16:54:41,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-03-20 16:54:41,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-03-20 16:54:41,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-03-20 16:54:41,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-03-20 16:54:41,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-03-20 16:54:41,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-03-20 16:54:41,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-03-20 16:54:41,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-03-20 16:54:41,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-03-20 16:54:41,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-03-20 16:54:41,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-03-20 16:54:41,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-03-20 16:54:41,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-03-20 16:54:41,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-03-20 16:54:41,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-03-20 16:54:41,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-03-20 16:54:41,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-03-20 16:54:41,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-03-20 16:54:41,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-03-20 16:54:41,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-03-20 16:54:41,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-03-20 16:54:41,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-03-20 16:54:41,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-03-20 16:54:41,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-03-20 16:54:41,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-03-20 16:54:41,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-03-20 16:54:41,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-03-20 16:54:41,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-03-20 16:54:41,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-03-20 16:54:41,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-03-20 16:54:41,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-03-20 16:54:41,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-03-20 16:54:41,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-03-20 16:54:41,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-03-20 16:54:41,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-03-20 16:54:41,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-03-20 16:54:41,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-03-20 16:54:41,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-03-20 16:54:41,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-03-20 16:54:41,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-03-20 16:54:41,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-03-20 16:54:41,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-03-20 16:54:41,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-03-20 16:54:41,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-03-20 16:54:41,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-03-20 16:54:41,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-03-20 16:54:42,071 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0077881924820476555); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.010314906998965449); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0077881924820476555); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-20 16:54:42,071 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-03-20 16:54:42,071 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=7.816571970559003E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0077881924820476555); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.010314906998965449); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0077881924820476555); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-03-20 16:54:42,325 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 379 ms to try 5000 different iterations.  Found a solution that moves 307 regions; Going from a computed imbalance of 7.816571970559003E-4 to a new imbalance of 3.8480766259450975E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.694119368783758E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0024618298195866546); MoveCostFunction : (multiplier=7.0, imbalance=0.03411111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.694119368783758E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-20 16:54:42,326 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1161962706:83 , srv1524117808:83 , srv1642334392:83 , srv1715035624:83 , srv463910070:83 , srv1157103363:84 , srv1329731745:84 , srv1482958952:84 , srv1493039153:84 , srv158086359:84 , srv1735503112:84 , srv1739560364:84 , srv1769212565:84 , srv1799731772:84 , srv1816034785:84 , srv1825434873:84 , srv1838249150:84 , srv1911404740:84 , srv1946892778:84 , srv195157664:84 , srv2037512389:84 , srv2049238365:84 , srv2060581181:84 , srv2080756296:84 , srv208278686:84 , srv2087294807:84 , srv2116471643:84 , srv2134022843:84 , srv2138366135:84 , srv305085847:84 , srv380459719:84 , srv387160648:84 , srv390390511:84 , srv450057666:84 , srv659851472:84 , srv683924424:84 , srv718604911:84 , srv728826233:84 , srv76969324:84 , srv853380171:84 , srv927909894:84 , srv1010636852:85 , srv1075228465:85 , srv1082070072:85 , srv1126701451:85 , srv1179796924:85 , srv1263085573:85 , srv1318334870:85 , srv1332938545:85 , srv1395594923:85 , srv1418280300:85 , srv1518465775:85 , srv1530397171:85 , srv1540884956:85 , srv1648308996:85 , srv1688566269:85 , srv1721390741:85 , srv1780020215:85 , srv1788202218:85 , srv1805255051:85 , srv1830098496:85 , srv1833475319:85 , srv1984687355:85 , srv1994000141:85 , srv2043823485:85 , srv205030921:85 , srv223495324:85 , srv270629074:85 , srv395131930:85 , srv402923580:85 , srv444338984:85 , srv459588057:85 , srv468232300:85 , srv485296249:85 , srv514014712:85 , srv564227331:85 , srv606020100:85 , srv628626359:85 , srv649323056:85 , srv684755254:85 , srv709481546:85 , srv8400371:85 , srv846821884:85 , srv889065435:85 , srv1140845359:86 , srv124529410:86 , srv1337952446:86 , srv1381080443:86 , srv1381973416:86 , srv1582204142:86 , srv1729135098:86 , srv1744117989:86 , srv1927614527:86 , srv58538325:86 , srv612351158:86 , srv727478809:86 , srv754929532:86 , srv786210135:86 , srv2069596160:87 , srv404791997:615 }
2024-03-20 16:54:42,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-03-20 16:54:42,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1493039153=19, srv709481546=88, srv1830098496=42, srv158086359=24, srv387160648=67, srv1946892778=47, srv728826233=91, srv1642334392=26, srv1395594923=16, srv1838249150=44, srv1161962706=6, srv2116471643=60, srv683924424=86, srv1715035624=29, srv606020100=81, srv659851472=85, srv1524117808=21, srv1518465775=20, srv1648308996=27, srv58538325=80, srv684755254=87, srv754929532=92, srv1582204142=25, srv195157664=48, srv727478809=90, srv2060581181=55, srv1381973416=15, srv2049238365=53, srv404791997=71, srv76969324=93, srv270629074=64, srv1994000141=50, srv649323056=84, srv2087294807=59, srv1126701451=3, srv468232300=76, srv2043823485=52, srv514014712=78, srv889065435=98, srv1140845359=4, srv205030921=54, srv1381080443=14, srv1729135098=31, srv2134022843=61, srv1911404740=45, srv2037512389=51, srv444338984=72, srv1744117989=34, srv1825434873=41, srv1329731745=11, srv1530397171=22, srv1805255051=39, srv2138366135=62, srv1418280300=17, srv208278686=58, srv485296249=77, srv628626359=83, srv927909894=99, srv1318334870=10, srv1735503112=32, srv390390511=68, srv1799731772=38, srv380459719=66, srv1482958952=18, srv459588057=74, srv853380171=97, srv1263085573=9, srv1984687355=49, srv402923580=70, srv786210135=94, srv1179796924=7, srv1739560364=33, srv1927614527=46, srv718604911=89, srv1780020215=36, srv223495324=63, srv846821884=96, srv612351158=82, srv395131930=69, srv2069596160=56, srv1337952446=13, srv1082070072=2, srv450057666=73, srv1332938545=12, srv1010636852=0, srv564227331=79, srv1157103363=5, srv124529410=8, srv1075228465=1, srv1788202218=37, srv305085847=65, srv463910070=75, srv1769212565=35, srv1816034785=40, srv8400371=95, srv1721390741=30, srv1833475319=43, srv1540884956=23, srv1688566269=28, srv2080756296=57} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-03-20 16:54:42,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-03-20 16:54:42,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-03-20 16:54:42,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-03-20 16:54:42,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-03-20 16:54:42,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-03-20 16:54:42,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-03-20 16:54:42,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-03-20 16:54:42,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-03-20 16:54:42,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-03-20 16:54:42,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-03-20 16:54:42,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-03-20 16:54:42,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-03-20 16:54:42,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-03-20 16:54:42,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-03-20 16:54:42,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-03-20 16:54:42,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-03-20 16:54:42,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-03-20 16:54:42,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-03-20 16:54:42,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-03-20 16:54:42,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-03-20 16:54:42,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-03-20 16:54:42,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-03-20 16:54:42,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-03-20 16:54:42,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-03-20 16:54:42,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-03-20 16:54:42,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-03-20 16:54:42,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-03-20 16:54:42,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-03-20 16:54:42,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-03-20 16:54:42,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-03-20 16:54:42,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-03-20 16:54:42,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-03-20 16:54:42,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-03-20 16:54:42,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-03-20 16:54:42,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-03-20 16:54:42,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-03-20 16:54:42,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-03-20 16:54:42,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-03-20 16:54:42,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-03-20 16:54:42,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-03-20 16:54:42,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-03-20 16:54:42,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-03-20 16:54:42,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-03-20 16:54:42,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-03-20 16:54:42,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-03-20 16:54:42,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-03-20 16:54:42,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-03-20 16:54:42,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-03-20 16:54:42,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-03-20 16:54:42,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-03-20 16:54:42,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-03-20 16:54:42,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-03-20 16:54:42,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-03-20 16:54:42,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-03-20 16:54:42,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-03-20 16:54:42,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-03-20 16:54:42,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-03-20 16:54:42,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-03-20 16:54:42,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-03-20 16:54:42,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-03-20 16:54:42,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-03-20 16:54:42,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-03-20 16:54:42,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-03-20 16:54:42,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-03-20 16:54:42,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-03-20 16:54:42,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-03-20 16:54:42,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-03-20 16:54:42,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-03-20 16:54:42,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-03-20 16:54:42,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-03-20 16:54:42,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-03-20 16:54:42,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-03-20 16:54:42,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-03-20 16:54:42,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-03-20 16:54:42,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-03-20 16:54:42,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-03-20 16:54:42,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-03-20 16:54:42,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-03-20 16:54:42,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-03-20 16:54:42,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-03-20 16:54:42,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-03-20 16:54:42,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-03-20 16:54:42,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-03-20 16:54:42,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-03-20 16:54:42,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-03-20 16:54:42,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-03-20 16:54:42,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-03-20 16:54:42,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-03-20 16:54:42,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-03-20 16:54:42,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-03-20 16:54:42,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-03-20 16:54:42,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-03-20 16:54:42,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-03-20 16:54:42,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-03-20 16:54:42,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-03-20 16:54:42,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-03-20 16:54:42,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-03-20 16:54:42,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-03-20 16:54:42,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-03-20 16:54:42,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-03-20 16:54:42,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-03-20 16:54:42,504 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.694119368783758E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0024618298195866546); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.694119368783758E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-20 16:54:42,504 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-03-20 16:54:42,504 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=3.6331106313847674E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.694119368783758E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0024618298195866546); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.694119368783758E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-03-20 16:54:42,765 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 380 ms to try 5000 different iterations.  Found a solution that moves 42 regions; Going from a computed imbalance of 3.6331106313847674E-5 to a new imbalance of 8.553854271709118E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.004666666666666667); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-20 16:54:42,765 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1381080443:83 , srv1582204142:83 , srv1735503112:83 , srv1744117989:83 , srv2069596160:83 , srv444338984:83 , srv468232300:83 , srv514014712:83 , srv58538325:83 , srv683924424:83 , srv786210135:83 , srv846821884:83 , srv1010636852:84 , srv1075228465:84 , srv1082070072:84 , srv1126701451:84 , srv1140845359:84 , srv1157103363:84 , srv1161962706:84 , srv1179796924:84 , srv124529410:84 , srv1263085573:84 , srv1318334870:84 , srv1329731745:84 , srv1332938545:84 , srv1337952446:84 , srv1381973416:84 , srv1395594923:84 , srv1418280300:84 , srv1482958952:84 , srv1493039153:84 , srv1518465775:84 , srv1524117808:84 , srv1530397171:84 , srv1540884956:84 , srv158086359:84 , srv1642334392:84 , srv1648308996:84 , srv1688566269:84 , srv1715035624:84 , srv1721390741:84 , srv1729135098:84 , srv1739560364:84 , srv1780020215:84 , srv1788202218:84 , srv1805255051:84 , srv1816034785:84 , srv1825434873:84 , srv1830098496:84 , srv1833475319:84 , srv1838249150:84 , srv1911404740:84 , srv1927614527:84 , srv195157664:84 , srv1984687355:84 , srv1994000141:84 , srv2037512389:84 , srv2043823485:84 , srv2049238365:84 , srv205030921:84 , srv2080756296:84 , srv208278686:84 , srv2116471643:84 , srv2134022843:84 , srv2138366135:84 , srv223495324:84 , srv270629074:84 , srv305085847:84 , srv380459719:84 , srv390390511:84 , srv395131930:84 , srv402923580:84 , srv450057666:84 , srv459588057:84 , srv485296249:84 , srv564227331:84 , srv606020100:84 , srv612351158:84 , srv628626359:84 , srv649323056:84 , srv684755254:84 , srv709481546:84 , srv718604911:84 , srv727478809:84 , srv754929532:84 , srv8400371:84 , srv853380171:84 , srv889065435:84 , srv1769212565:85 , srv1799731772:85 , srv1946892778:85 , srv2060581181:85 , srv2087294807:85 , srv387160648:85 , srv463910070:85 , srv659851472:85 , srv76969324:85 , srv927909894:85 , srv728826233:86 , srv404791997:684 }
2024-03-20 16:54:42,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-03-20 16:54:42,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1493039153=19, srv709481546=88, srv1830098496=42, srv158086359=24, srv387160648=67, srv1946892778=47, srv728826233=91, srv1642334392=26, srv1395594923=16, srv1838249150=44, srv1161962706=6, srv2116471643=60, srv683924424=86, srv1715035624=29, srv606020100=81, srv659851472=85, srv1524117808=21, srv1518465775=20, srv1648308996=27, srv58538325=80, srv684755254=87, srv754929532=92, srv1582204142=25, srv195157664=48, srv727478809=90, srv2060581181=55, srv1381973416=15, srv2049238365=53, srv404791997=71, srv76969324=93, srv270629074=64, srv1994000141=50, srv649323056=84, srv2087294807=59, srv1126701451=3, srv468232300=76, srv2043823485=52, srv514014712=78, srv889065435=98, srv1140845359=4, srv205030921=54, srv1381080443=14, srv1729135098=31, srv2134022843=61, srv1911404740=45, srv2037512389=51, srv444338984=72, srv1744117989=34, srv1825434873=41, srv1329731745=11, srv1530397171=22, srv1805255051=39, srv2138366135=62, srv1418280300=17, srv208278686=58, srv485296249=77, srv628626359=83, srv927909894=99, srv1318334870=10, srv1735503112=32, srv390390511=68, srv1799731772=38, srv380459719=66, srv1482958952=18, srv459588057=74, srv853380171=97, srv1263085573=9, srv1984687355=49, srv402923580=70, srv786210135=94, srv1179796924=7, srv1739560364=33, srv1927614527=46, srv718604911=89, srv1780020215=36, srv223495324=63, srv846821884=96, srv612351158=82, srv395131930=69, srv2069596160=56, srv1337952446=13, srv1082070072=2, srv450057666=73, srv1332938545=12, srv1010636852=0, srv564227331=79, srv1157103363=5, srv124529410=8, srv1075228465=1, srv1788202218=37, srv305085847=65, srv463910070=75, srv1769212565=35, srv1816034785=40, srv8400371=95, srv1721390741=30, srv1833475319=43, srv1540884956=23, srv1688566269=28, srv2080756296=57} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-03-20 16:54:42,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-03-20 16:54:42,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-03-20 16:54:42,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-03-20 16:54:42,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-03-20 16:54:42,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-03-20 16:54:42,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-03-20 16:54:42,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-03-20 16:54:42,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-03-20 16:54:42,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-03-20 16:54:42,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-03-20 16:54:42,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-03-20 16:54:42,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-03-20 16:54:42,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-03-20 16:54:42,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-03-20 16:54:42,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-03-20 16:54:42,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-03-20 16:54:42,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-03-20 16:54:42,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-03-20 16:54:42,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-03-20 16:54:42,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-03-20 16:54:42,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-03-20 16:54:42,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-03-20 16:54:42,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-03-20 16:54:42,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-03-20 16:54:42,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-03-20 16:54:42,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-03-20 16:54:42,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-03-20 16:54:42,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-03-20 16:54:42,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-03-20 16:54:42,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-03-20 16:54:42,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-03-20 16:54:42,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-03-20 16:54:42,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-03-20 16:54:42,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-03-20 16:54:42,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-03-20 16:54:42,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-03-20 16:54:42,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-03-20 16:54:42,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-03-20 16:54:42,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-03-20 16:54:42,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-03-20 16:54:42,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-03-20 16:54:42,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-03-20 16:54:42,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-03-20 16:54:42,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-03-20 16:54:42,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-03-20 16:54:42,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-03-20 16:54:42,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-03-20 16:54:42,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-03-20 16:54:42,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-03-20 16:54:42,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-03-20 16:54:42,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-03-20 16:54:42,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-03-20 16:54:42,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-03-20 16:54:42,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-03-20 16:54:42,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-03-20 16:54:42,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-03-20 16:54:42,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-03-20 16:54:42,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-03-20 16:54:42,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-03-20 16:54:42,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-03-20 16:54:42,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-03-20 16:54:42,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-03-20 16:54:42,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-03-20 16:54:42,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-03-20 16:54:42,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-03-20 16:54:42,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-03-20 16:54:42,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-03-20 16:54:42,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-03-20 16:54:42,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-03-20 16:54:42,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-03-20 16:54:42,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-03-20 16:54:42,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-03-20 16:54:42,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-03-20 16:54:42,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-03-20 16:54:42,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-03-20 16:54:42,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-03-20 16:54:42,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-03-20 16:54:42,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-03-20 16:54:42,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-03-20 16:54:42,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-03-20 16:54:42,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-03-20 16:54:42,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-03-20 16:54:42,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-03-20 16:54:42,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-03-20 16:54:42,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-03-20 16:54:42,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-03-20 16:54:42,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-03-20 16:54:42,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-03-20 16:54:42,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-03-20 16:54:42,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-03-20 16:54:42,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-03-20 16:54:42,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-03-20 16:54:42,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-03-20 16:54:42,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-03-20 16:54:42,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-03-20 16:54:42,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-03-20 16:54:42,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-03-20 16:54:42,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-03-20 16:54:42,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-03-20 16:54:42,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-03-20 16:54:42,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-03-20 16:54:42,939 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=8.259763985991403E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-20 16:54:42,939 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1381080443:83 , srv1582204142:83 , srv1735503112:83 , srv1744117989:83 , srv2069596160:83 , srv444338984:83 , srv468232300:83 , srv514014712:83 , srv58538325:83 , srv683924424:83 , srv786210135:83 , srv846821884:83 , srv1010636852:84 , srv1075228465:84 , srv1082070072:84 , srv1126701451:84 , srv1140845359:84 , srv1157103363:84 , srv1161962706:84 , srv1179796924:84 , srv124529410:84 , srv1263085573:84 , srv1318334870:84 , srv1329731745:84 , srv1332938545:84 , srv1337952446:84 , srv1381973416:84 , srv1395594923:84 , srv1418280300:84 , srv1482958952:84 , srv1493039153:84 , srv1518465775:84 , srv1524117808:84 , srv1530397171:84 , srv1540884956:84 , srv158086359:84 , srv1642334392:84 , srv1648308996:84 , srv1688566269:84 , srv1715035624:84 , srv1721390741:84 , srv1729135098:84 , srv1739560364:84 , srv1780020215:84 , srv1788202218:84 , srv1805255051:84 , srv1816034785:84 , srv1825434873:84 , srv1830098496:84 , srv1833475319:84 , srv1838249150:84 , srv1911404740:84 , srv1927614527:84 , srv195157664:84 , srv1984687355:84 , srv1994000141:84 , srv2037512389:84 , srv2043823485:84 , srv2049238365:84 , srv205030921:84 , srv2080756296:84 , srv208278686:84 , srv2116471643:84 , srv2134022843:84 , srv2138366135:84 , srv223495324:84 , srv270629074:84 , srv305085847:84 , srv380459719:84 , srv390390511:84 , srv395131930:84 , srv402923580:84 , srv450057666:84 , srv459588057:84 , srv485296249:84 , srv564227331:84 , srv606020100:84 , srv612351158:84 , srv628626359:84 , srv649323056:84 , srv684755254:84 , srv709481546:84 , srv718604911:84 , srv727478809:84 , srv754929532:84 , srv8400371:84 , srv853380171:84 , srv889065435:84 , srv1769212565:85 , srv1799731772:85 , srv1946892778:85 , srv2060581181:85 , srv2087294807:85 , srv387160648:85 , srv463910070:85 , srv659851472:85 , srv76969324:85 , srv927909894:85 , srv728826233:86 , srv404791997:684 }
2024-03-20 16:54:42,992 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=617 (was 610) - SystemLoadAverage LEAK? -, ProcessCount=196 (was 193) - ProcessCount LEAK? -, AvailableMemoryMB=2356 (was 3007)
2024-03-20 16:54:43,008 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=617, ProcessCount=196, AvailableMemoryMB=2355
2024-03-20 16:54:43,009 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-03-20 16:54:43,009 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2024-03-20 16:54:43,025 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1057631379:84 , srv1146132141:84 , srv1187266378:84 , srv1244149204:84 , srv1287479994:84 , srv1449899124:84 , srv1465442909:84 , srv1540711675:84 , srv1625372812:84 , srv1702824893:84 , srv1754191059:84 , srv1786191955:84 , srv1900489336:84 , srv1919633890:84 , srv1940897807:84 , srv1961633439:84 , srv1983011521:84 , srv1997151891:84 , srv1998463999:84 , srv204698980:84 , srv2103524228:84 , srv2119775477:84 , srv40834619:84 , srv421125392:84 , srv623810338:84 , srv6468468:84 , srv656693855:84 , srv86076292:84 , srv943489979:84 , srv1370286909:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2024-03-20 16:54:43,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-03-20 16:54:43,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1625372812=9, srv1983011521=17, srv40834619=23, srv1786191955=12, srv1997151891=18, srv86076292=28, srv1702824893=10, srv1540711675=8, srv1961633439=16, srv1919633890=14, srv2119775477=22, srv1465442909=7, srv1370286909=5, srv1057631379=0, srv1900489336=13, srv421125392=24, srv1187266378=2, srv1449899124=6, srv623810338=25, srv943489979=29, srv1940897807=15, srv204698980=20, srv2103524228=21, srv1146132141=1, srv1287479994=4, srv656693855=27, srv1754191059=11, srv1244149204=3, srv1998463999=19, srv6468468=26} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-03-20 16:54:43,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-03-20 16:54:43,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-03-20 16:54:43,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-03-20 16:54:43,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-03-20 16:54:43,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-03-20 16:54:43,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-03-20 16:54:43,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-03-20 16:54:43,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-03-20 16:54:43,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-03-20 16:54:43,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-03-20 16:54:43,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-03-20 16:54:43,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-03-20 16:54:43,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-03-20 16:54:43,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-03-20 16:54:43,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-03-20 16:54:43,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-03-20 16:54:43,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-03-20 16:54:43,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-03-20 16:54:43,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-03-20 16:54:43,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-03-20 16:54:43,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-03-20 16:54:43,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-03-20 16:54:43,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-03-20 16:54:43,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-03-20 16:54:43,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-03-20 16:54:43,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-03-20 16:54:43,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-03-20 16:54:43,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-03-20 16:54:43,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-03-20 16:54:43,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-03-20 16:54:43,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-03-20 16:54:43,075 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1008633164615855, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-20 16:54:43,075 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.991251086028096, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1008633164615855, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2024-03-20 16:56:43,056 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6221906 different iterations.  Found a solution that moves 1856 regions; Going from a computed imbalance of 0.991251086028096 to a new imbalance of 4.3319965896196796E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6874074074074074, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-20 16:56:43,069 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1057631379:90 , srv1146132141:90 , srv1187266378:90 , srv1244149204:90 , srv1287479994:90 , srv1370286909:90 , srv1449899124:90 , srv1465442909:90 , srv1540711675:90 , srv1625372812:90 , srv1702824893:90 , srv1754191059:90 , srv1786191955:90 , srv1900489336:90 , srv1919633890:90 , srv1940897807:90 , srv1961633439:90 , srv1983011521:90 , srv1997151891:90 , srv1998463999:90 , srv204698980:90 , srv2103524228:90 , srv2119775477:90 , srv40834619:90 , srv421125392:90 , srv623810338:90 , srv6468468:90 , srv656693855:90 , srv86076292:90 , srv943489979:90 }
2024-03-20 16:56:43,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-03-20 16:56:43,072 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1625372812=9, srv1983011521=17, srv40834619=23, srv1786191955=12, srv1997151891=18, srv86076292=28, srv1702824893=10, srv1540711675=8, srv1961633439=16, srv1919633890=14, srv2119775477=22, srv1465442909=7, srv1370286909=5, srv1057631379=0, srv1900489336=13, srv421125392=24, srv1187266378=2, srv1449899124=6, srv623810338=25, srv943489979=29, srv1940897807=15, srv204698980=20, srv2103524228=21, srv1146132141=1, srv1287479994=4, srv656693855=27, srv1754191059=11, srv1244149204=3, srv1998463999=19, srv6468468=26} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-03-20 16:56:43,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-03-20 16:56:43,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-03-20 16:56:43,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-03-20 16:56:43,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-03-20 16:56:43,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-03-20 16:56:43,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-03-20 16:56:43,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-03-20 16:56:43,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-03-20 16:56:43,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-03-20 16:56:43,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-03-20 16:56:43,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-03-20 16:56:43,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-03-20 16:56:43,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-03-20 16:56:43,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-03-20 16:56:43,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-03-20 16:56:43,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-03-20 16:56:43,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-03-20 16:56:43,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-03-20 16:56:43,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-03-20 16:56:43,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-03-20 16:56:43,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-03-20 16:56:43,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-03-20 16:56:43,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-03-20 16:56:43,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-03-20 16:56:43,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-03-20 16:56:43,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-03-20 16:56:43,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-03-20 16:56:43,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-03-20 16:56:43,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-03-20 16:56:43,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-03-20 16:56:43,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-03-20 16:56:43,097 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-20 16:56:43,098 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1057631379:90 , srv1146132141:90 , srv1187266378:90 , srv1244149204:90 , srv1287479994:90 , srv1370286909:90 , srv1449899124:90 , srv1465442909:90 , srv1540711675:90 , srv1625372812:90 , srv1702824893:90 , srv1754191059:90 , srv1786191955:90 , srv1900489336:90 , srv1919633890:90 , srv1940897807:90 , srv1961633439:90 , srv1983011521:90 , srv1997151891:90 , srv1998463999:90 , srv204698980:90 , srv2103524228:90 , srv2119775477:90 , srv40834619:90 , srv421125392:90 , srv623810338:90 , srv6468468:90 , srv656693855:90 , srv86076292:90 , srv943489979:90 }
2024-03-20 16:56:43,123 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=860 (was 617) - SystemLoadAverage LEAK? -, ProcessCount=198 (was 196) - ProcessCount LEAK? -, AvailableMemoryMB=3396 (was 2355) - AvailableMemoryMB LEAK? -