2024-02-05 02:53:11,261 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2024-02-05 02:53:11,552 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2024-02-05 02:53:11,667 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2024-02-05 02:53:11,733 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-02-05 02:53:11,742 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-02-05 02:53:11,757 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=284, ProcessCount=169, AvailableMemoryMB=3214
2024-02-05 02:53:11,759 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-02-05 02:53:11,759 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-02-05 02:53:11,875 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1009236336:84 , srv1042480651:84 , srv1054393396:84 , srv1061184343:84 , srv1099859862:84 , srv1118336637:84 , srv1177891322:84 , srv1182953725:84 , srv1214843210:84 , srv1236284563:84 , srv1264468192:84 , srv1265062165:84 , srv1272681439:84 , srv1273519014:84 , srv1285009451:84 , srv1290108101:84 , srv1291080411:84 , srv1311579464:84 , srv1328471885:84 , srv133609377:84 , srv136055579:84 , srv1378768350:84 , srv1378835569:84 , srv1379637598:84 , srv1382153121:84 , srv1389869859:84 , srv1404176955:84 , srv1439047171:84 , srv1447956787:84 , srv1458831965:84 , srv1461906324:84 , srv1467549296:84 , srv1473415808:84 , srv1550517029:84 , srv1563770918:84 , srv1569114821:84 , srv1595847305:84 , srv1608877635:84 , srv1615495159:84 , srv1618029376:84 , srv1645009488:84 , srv1674835218:84 , srv1680493983:84 , srv1685284687:84 , srv1780015094:84 , srv1797593631:84 , srv180334465:84 , srv1848776723:84 , srv1853426961:84 , srv1857346222:84 , srv1872019917:84 , srv1895072614:84 , srv1950909052:84 , srv199796600:84 , srv2011300067:84 , srv2016516989:84 , srv2045399544:84 , srv205125390:84 , srv22485517:84 , srv22936862:84 , srv2497583:84 , srv255968183:84 , srv257125741:84 , srv280375379:84 , srv298649200:84 , srv301456363:84 , srv305702596:84 , srv315098169:84 , srv322580374:84 , srv355963012:84 , srv363734574:84 , srv365372721:84 , srv373399525:84 , srv404588942:84 , srv477634312:84 , srv480994044:84 , srv505933894:84 , srv506563563:84 , srv530010161:84 , srv55306089:84 , srv564510362:84 , srv573552943:84 , srv582038221:84 , srv594371146:84 , srv613345306:84 , srv617430806:84 , srv628096652:84 , srv646767629:84 , srv649478036:84 , srv70994355:84 , srv721167113:84 , srv773628487:84 , srv786990502:84 , srv789190436:84 , srv803764986:84 , srv839310002:84 , srv871907884:84 , srv943737694:84 , srv997735341:84 , srv242511437:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2024-02-05 02:53:11,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-05 02:53:11,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1273519014=13, srv205125390=57, srv839310002=96, srv1473415808=32, srv1674835218=41, srv2045399544=56, srv1236284563=9, srv1272681439=12, srv786990502=93, srv1569114821=35, srv280375379=64, srv1264468192=10, srv505933894=77, srv2497583=61, srv2016516989=55, srv1378768350=21, srv257125741=63, srv1061184343=3, srv373399525=73, srv1042480651=1, srv1378835569=22, srv997735341=99, srv613345306=85, srv1857346222=49, srv355963012=70, srv1118336637=5, srv1439047171=27, srv322580374=69, srv365372721=72, srv1404176955=26, srv1950909052=52, srv1608877635=37, srv180334465=46, srv871907884=97, srv2011300067=54, srv594371146=84, srv1099859862=4, srv315098169=68, srv363734574=71, srv477634312=75, srv1895072614=51, srv22936862=59, srv22485517=58, srv1615495159=38, srv582038221=83, srv480994044=76, srv943737694=98, srv255968183=62, srv1054393396=2, srv1680493983=42, srv789190436=94, srv404588942=74, srv70994355=90, srv1780015094=44, srv649478036=89, srv1291080411=16, srv1550517029=33, srv506563563=78, srv1645009488=40, srv133609377=19, srv1182953725=7, srv628096652=87, srv1389869859=25, srv1618029376=39, srv55306089=80, srv1685284687=43, srv721167113=91, srv1177891322=6, srv1379637598=23, srv301456363=66, srv136055579=20, srv1872019917=50, srv242511437=60, srv1595847305=36, srv1009236336=0, srv305702596=67, srv1328471885=18, srv1461906324=30, srv1853426961=48, srv773628487=92, srv1214843210=8, srv1467549296=31, srv199796600=53, srv1797593631=45, srv646767629=88, srv564510362=81, srv1458831965=29, srv1563770918=34, srv530010161=79, srv1311579464=17, srv298649200=65, srv1265062165=11, srv1447956787=28, srv1285009451=14, srv1290108101=15, srv617430806=86, srv1382153121=24, srv1848776723=47, srv573552943=82, srv803764986=95} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-05 02:53:11,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-05 02:53:11,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-05 02:53:11,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-05 02:53:11,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-05 02:53:11,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-05 02:53:11,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-05 02:53:11,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-05 02:53:11,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-05 02:53:11,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-05 02:53:11,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-05 02:53:11,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-05 02:53:11,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-05 02:53:11,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-05 02:53:11,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-05 02:53:11,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-05 02:53:11,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-05 02:53:11,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-05 02:53:11,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-05 02:53:11,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-05 02:53:11,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-05 02:53:11,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-05 02:53:11,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-05 02:53:11,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-05 02:53:11,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-05 02:53:11,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-05 02:53:11,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-05 02:53:11,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-05 02:53:11,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-05 02:53:11,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-05 02:53:11,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-05 02:53:11,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-02-05 02:53:11,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-02-05 02:53:11,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-02-05 02:53:11,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-02-05 02:53:11,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-02-05 02:53:11,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-02-05 02:53:11,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-02-05 02:53:11,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-02-05 02:53:11,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-02-05 02:53:11,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-02-05 02:53:11,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-02-05 02:53:11,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-02-05 02:53:11,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-02-05 02:53:11,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-02-05 02:53:11,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-02-05 02:53:11,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-02-05 02:53:11,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-02-05 02:53:11,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-02-05 02:53:11,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-02-05 02:53:11,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-02-05 02:53:11,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-02-05 02:53:11,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-02-05 02:53:11,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-02-05 02:53:11,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-02-05 02:53:11,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-02-05 02:53:11,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-02-05 02:53:11,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-02-05 02:53:11,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-02-05 02:53:11,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-02-05 02:53:11,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-02-05 02:53:11,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-02-05 02:53:11,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-02-05 02:53:11,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-02-05 02:53:11,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-02-05 02:53:11,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-02-05 02:53:11,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-02-05 02:53:11,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-02-05 02:53:11,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-02-05 02:53:11,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-02-05 02:53:11,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-02-05 02:53:11,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-02-05 02:53:11,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-02-05 02:53:11,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-02-05 02:53:11,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-02-05 02:53:11,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-02-05 02:53:11,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-02-05 02:53:11,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-02-05 02:53:11,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-02-05 02:53:11,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-02-05 02:53:11,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-02-05 02:53:11,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-02-05 02:53:11,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-02-05 02:53:11,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-02-05 02:53:11,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-02-05 02:53:11,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-02-05 02:53:11,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-02-05 02:53:11,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-02-05 02:53:11,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-02-05 02:53:11,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-02-05 02:53:11,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-02-05 02:53:11,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-02-05 02:53:11,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-02-05 02:53:11,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-02-05 02:53:11,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-02-05 02:53:11,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-02-05 02:53:11,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-02-05 02:53:11,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-02-05 02:53:11,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-02-05 02:53:11,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-02-05 02:53:11,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-02-05 02:53:11,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-02-05 02:53:12,228 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-05 02:53:12,228 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-02-05 02:53:12,229 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-02-05 02:53:12,573 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 560 ms to try 5000 different iterations.  Found a solution that moves 4707 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17544676193759542. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04100465483842969, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05666963747411669, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.523, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04100465483842969, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17066666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23675, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-05 02:53:12,588 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1467549296:81 , srv1848776723:81 , srv1872019917:81 , srv613345306:81 , srv1061184343:82 , srv1285009451:82 , srv1311579464:82 , srv1328471885:82 , srv1680493983:82 , srv180334465:82 , srv2016516989:82 , srv2497583:82 , srv404588942:82 , srv803764986:82 , srv1290108101:83 , srv1389869859:83 , srv1685284687:83 , srv22485517:83 , srv255968183:83 , srv573552943:83 , srv649478036:83 , srv871907884:83 , srv1042480651:84 , srv1272681439:84 , srv1595847305:84 , srv1608877635:84 , srv1618029376:84 , srv1645009488:84 , srv205125390:84 , srv564510362:84 , srv789190436:84 , srv997735341:84 , srv1379637598:85 , srv1404176955:85 , srv1461906324:85 , srv1950909052:85 , srv2011300067:85 , srv22936862:85 , srv280375379:85 , srv305702596:85 , srv322580374:85 , srv363734574:85 , srv480994044:85 , srv721167113:85 , srv786990502:85 , srv1177891322:86 , srv1182953725:86 , srv1857346222:86 , srv301456363:86 , srv55306089:86 , srv617430806:86 , srv70994355:86 , srv839310002:86 , srv1382153121:87 , srv1439047171:87 , srv1563770918:87 , srv1674835218:87 , srv373399525:87 , srv477634312:87 , srv594371146:87 , srv1009236336:88 , srv1236284563:88 , srv1264468192:88 , srv1473415808:88 , srv1550517029:88 , srv1853426961:88 , srv1895072614:88 , srv298649200:88 , srv505933894:88 , srv506563563:88 , srv628096652:88 , srv943737694:88 , srv1118336637:89 , srv1214843210:89 , srv133609377:89 , srv1378835569:89 , srv1458831965:89 , srv1615495159:89 , srv257125741:89 , srv365372721:89 , srv582038221:89 , srv1054393396:90 , srv1099859862:90 , srv1265062165:90 , srv1273519014:90 , srv1291080411:90 , srv136055579:90 , srv1378768350:90 , srv1780015094:90 , srv1569114821:91 , srv199796600:91 , srv315098169:91 , srv646767629:91 , srv1447956787:92 , srv2045399544:92 , srv530010161:92 , srv773628487:92 , srv355963012:93 , srv1797593631:94 , srv242511437:454 }
2024-02-05 02:53:12,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-05 02:53:12,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1273519014=13, srv205125390=57, srv839310002=96, srv1473415808=32, srv1674835218=41, srv2045399544=56, srv1236284563=9, srv1272681439=12, srv786990502=93, srv1569114821=35, srv280375379=64, srv1264468192=10, srv505933894=77, srv2497583=61, srv2016516989=55, srv1378768350=21, srv257125741=63, srv1061184343=3, srv373399525=73, srv1042480651=1, srv1378835569=22, srv997735341=99, srv613345306=85, srv1857346222=49, srv355963012=70, srv1118336637=5, srv1439047171=27, srv322580374=69, srv365372721=72, srv1404176955=26, srv1950909052=52, srv1608877635=37, srv180334465=46, srv871907884=97, srv2011300067=54, srv594371146=84, srv1099859862=4, srv315098169=68, srv363734574=71, srv477634312=75, srv1895072614=51, srv22936862=59, srv22485517=58, srv1615495159=38, srv582038221=83, srv480994044=76, srv943737694=98, srv255968183=62, srv1054393396=2, srv1680493983=42, srv789190436=94, srv404588942=74, srv70994355=90, srv1780015094=44, srv649478036=89, srv1291080411=16, srv1550517029=33, srv506563563=78, srv1645009488=40, srv133609377=19, srv1182953725=7, srv628096652=87, srv1389869859=25, srv1618029376=39, srv55306089=80, srv1685284687=43, srv721167113=91, srv1177891322=6, srv1379637598=23, srv301456363=66, srv136055579=20, srv1872019917=50, srv242511437=60, srv1595847305=36, srv1009236336=0, srv305702596=67, srv1328471885=18, srv1461906324=30, srv1853426961=48, srv773628487=92, srv1214843210=8, srv1467549296=31, srv199796600=53, srv1797593631=45, srv646767629=88, srv564510362=81, srv1458831965=29, srv1563770918=34, srv530010161=79, srv1311579464=17, srv298649200=65, srv1265062165=11, srv1447956787=28, srv1285009451=14, srv1290108101=15, srv617430806=86, srv1382153121=24, srv1848776723=47, srv573552943=82, srv803764986=95} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-05 02:53:12,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-05 02:53:12,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-05 02:53:12,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-05 02:53:12,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-05 02:53:12,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-05 02:53:12,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-05 02:53:12,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-05 02:53:12,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-05 02:53:12,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-05 02:53:12,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-05 02:53:12,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-05 02:53:12,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-05 02:53:12,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-05 02:53:12,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-05 02:53:12,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-05 02:53:12,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-05 02:53:12,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-05 02:53:12,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-05 02:53:12,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-05 02:53:12,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-05 02:53:12,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-05 02:53:12,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-05 02:53:12,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-05 02:53:12,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-05 02:53:12,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-05 02:53:12,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-05 02:53:12,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-05 02:53:12,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-05 02:53:12,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-05 02:53:12,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-05 02:53:12,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-02-05 02:53:12,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-02-05 02:53:12,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-02-05 02:53:12,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-02-05 02:53:12,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-02-05 02:53:12,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-02-05 02:53:12,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-02-05 02:53:12,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-02-05 02:53:12,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-02-05 02:53:12,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-02-05 02:53:12,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-02-05 02:53:12,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-02-05 02:53:12,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-02-05 02:53:12,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-02-05 02:53:12,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-02-05 02:53:12,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-02-05 02:53:12,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-02-05 02:53:12,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-02-05 02:53:12,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-02-05 02:53:12,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-02-05 02:53:12,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-02-05 02:53:12,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-02-05 02:53:12,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-02-05 02:53:12,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-02-05 02:53:12,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-02-05 02:53:12,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-02-05 02:53:12,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-02-05 02:53:12,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-02-05 02:53:12,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-02-05 02:53:12,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-02-05 02:53:12,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-02-05 02:53:12,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-02-05 02:53:12,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-02-05 02:53:12,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-02-05 02:53:12,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-02-05 02:53:12,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-02-05 02:53:12,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-02-05 02:53:12,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-02-05 02:53:12,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-02-05 02:53:12,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-02-05 02:53:12,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-02-05 02:53:12,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-02-05 02:53:12,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-02-05 02:53:12,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-02-05 02:53:12,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-02-05 02:53:12,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-02-05 02:53:12,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-02-05 02:53:12,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-02-05 02:53:12,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-02-05 02:53:12,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-02-05 02:53:12,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-02-05 02:53:12,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-02-05 02:53:12,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-02-05 02:53:12,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-02-05 02:53:12,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-02-05 02:53:12,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-02-05 02:53:12,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-02-05 02:53:12,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-02-05 02:53:12,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-02-05 02:53:12,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-02-05 02:53:12,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-02-05 02:53:12,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-02-05 02:53:12,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-02-05 02:53:12,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-02-05 02:53:12,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-02-05 02:53:12,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-02-05 02:53:12,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-02-05 02:53:12,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-02-05 02:53:12,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-02-05 02:53:12,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-02-05 02:53:12,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-02-05 02:53:12,810 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04100465483842969, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05666963747411669, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04100465483842969, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17066666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23675, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-05 02:53:12,810 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-02-05 02:53:12,811 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17541380281914606, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04100465483842969, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05666963747411669, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04100465483842969, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17066666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23675, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-02-05 02:53:13,058 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 394 ms to try 5000 different iterations.  Found a solution that moves 2653 regions; Going from a computed imbalance of 0.17541380281914606 to a new imbalance of 0.015573134242002289. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02363468487136436); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.030218062696528263, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2947777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02363468487136436); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012083333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.049166666666666664, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-05 02:53:13,068 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv355963012:78 , srv773628487:78 , srv1273519014:79 , srv1797593631:79 , srv1214843210:80 , srv1291080411:80 , srv199796600:80 , srv315098169:80 , srv530010161:80 , srv1569114821:81 , srv1780015094:81 , srv2045399544:81 , srv646767629:81 , srv1054393396:82 , srv1118336637:82 , srv133609377:82 , srv1378768350:82 , srv1447956787:82 , srv1615495159:82 , srv1853426961:82 , srv1895072614:82 , srv257125741:82 , srv1099859862:83 , srv136055579:83 , srv1458831965:83 , srv1473415808:83 , srv301456363:83 , srv1009236336:84 , srv1265062165:84 , srv1378835569:84 , srv1550517029:84 , srv1563770918:84 , srv1674835218:84 , srv1857346222:84 , srv298649200:84 , srv305702596:84 , srv365372721:84 , srv506563563:84 , srv582038221:84 , srv594371146:84 , srv943737694:84 , srv1264468192:85 , srv1379637598:85 , srv1404176955:85 , srv1439047171:85 , srv1950909052:85 , srv322580374:85 , srv477634312:85 , srv505933894:85 , srv628096652:85 , srv997735341:85 , srv1177891322:86 , srv1236284563:86 , srv1461906324:86 , srv1595847305:86 , srv1645009488:86 , srv2011300067:86 , srv280375379:86 , srv363734574:86 , srv55306089:86 , srv786990502:86 , srv839310002:86 , srv1042480651:87 , srv1382153121:87 , srv1389869859:87 , srv1618029376:87 , srv22485517:87 , srv480994044:87 , srv1182953725:88 , srv1290108101:88 , srv1685284687:88 , srv205125390:88 , srv22936862:88 , srv373399525:88 , srv404588942:88 , srv617430806:88 , srv649478036:88 , srv255968183:89 , srv573552943:89 , srv721167113:89 , srv789190436:89 , srv1285009451:90 , srv1608877635:90 , srv1848776723:90 , srv1872019917:90 , srv2016516989:90 , srv871907884:90 , srv1061184343:91 , srv1272681439:91 , srv1311579464:91 , srv1328471885:91 , srv1467549296:91 , srv1680493983:91 , srv2497583:91 , srv564510362:91 , srv180334465:92 , srv70994355:92 , srv803764986:92 , srv613345306:93 , srv242511437:530 }
2024-02-05 02:53:13,072 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-05 02:53:13,073 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1273519014=13, srv205125390=57, srv839310002=96, srv1473415808=32, srv1674835218=41, srv2045399544=56, srv1236284563=9, srv1272681439=12, srv786990502=93, srv1569114821=35, srv280375379=64, srv1264468192=10, srv505933894=77, srv2497583=61, srv2016516989=55, srv1378768350=21, srv257125741=63, srv1061184343=3, srv373399525=73, srv1042480651=1, srv1378835569=22, srv997735341=99, srv613345306=85, srv1857346222=49, srv355963012=70, srv1118336637=5, srv1439047171=27, srv322580374=69, srv365372721=72, srv1404176955=26, srv1950909052=52, srv1608877635=37, srv180334465=46, srv871907884=97, srv2011300067=54, srv594371146=84, srv1099859862=4, srv315098169=68, srv363734574=71, srv477634312=75, srv1895072614=51, srv22936862=59, srv22485517=58, srv1615495159=38, srv582038221=83, srv480994044=76, srv943737694=98, srv255968183=62, srv1054393396=2, srv1680493983=42, srv789190436=94, srv404588942=74, srv70994355=90, srv1780015094=44, srv649478036=89, srv1291080411=16, srv1550517029=33, srv506563563=78, srv1645009488=40, srv133609377=19, srv1182953725=7, srv628096652=87, srv1389869859=25, srv1618029376=39, srv55306089=80, srv1685284687=43, srv721167113=91, srv1177891322=6, srv1379637598=23, srv301456363=66, srv136055579=20, srv1872019917=50, srv242511437=60, srv1595847305=36, srv1009236336=0, srv305702596=67, srv1328471885=18, srv1461906324=30, srv1853426961=48, srv773628487=92, srv1214843210=8, srv1467549296=31, srv199796600=53, srv1797593631=45, srv646767629=88, srv564510362=81, srv1458831965=29, srv1563770918=34, srv530010161=79, srv1311579464=17, srv298649200=65, srv1265062165=11, srv1447956787=28, srv1285009451=14, srv1290108101=15, srv617430806=86, srv1382153121=24, srv1848776723=47, srv573552943=82, srv803764986=95} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-05 02:53:13,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-05 02:53:13,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-05 02:53:13,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-05 02:53:13,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-05 02:53:13,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-05 02:53:13,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-05 02:53:13,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-05 02:53:13,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-05 02:53:13,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-05 02:53:13,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-05 02:53:13,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-05 02:53:13,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-05 02:53:13,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-05 02:53:13,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-05 02:53:13,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-05 02:53:13,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-05 02:53:13,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-05 02:53:13,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-05 02:53:13,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-05 02:53:13,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-05 02:53:13,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-05 02:53:13,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-05 02:53:13,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-05 02:53:13,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-05 02:53:13,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-05 02:53:13,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-05 02:53:13,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-05 02:53:13,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-05 02:53:13,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-05 02:53:13,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-05 02:53:13,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-02-05 02:53:13,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-02-05 02:53:13,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-02-05 02:53:13,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-02-05 02:53:13,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-02-05 02:53:13,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-02-05 02:53:13,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-02-05 02:53:13,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-02-05 02:53:13,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-02-05 02:53:13,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-02-05 02:53:13,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-02-05 02:53:13,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-02-05 02:53:13,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-02-05 02:53:13,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-02-05 02:53:13,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-02-05 02:53:13,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-02-05 02:53:13,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-02-05 02:53:13,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-02-05 02:53:13,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-02-05 02:53:13,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-02-05 02:53:13,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-02-05 02:53:13,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-02-05 02:53:13,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-02-05 02:53:13,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-02-05 02:53:13,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-02-05 02:53:13,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-02-05 02:53:13,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-02-05 02:53:13,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-02-05 02:53:13,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-02-05 02:53:13,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-02-05 02:53:13,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-02-05 02:53:13,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-02-05 02:53:13,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-02-05 02:53:13,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-02-05 02:53:13,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-02-05 02:53:13,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-02-05 02:53:13,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-02-05 02:53:13,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-02-05 02:53:13,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-02-05 02:53:13,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-02-05 02:53:13,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-02-05 02:53:13,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-02-05 02:53:13,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-02-05 02:53:13,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-02-05 02:53:13,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-02-05 02:53:13,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-02-05 02:53:13,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-02-05 02:53:13,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-02-05 02:53:13,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-02-05 02:53:13,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-02-05 02:53:13,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-02-05 02:53:13,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-02-05 02:53:13,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-02-05 02:53:13,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-02-05 02:53:13,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-02-05 02:53:13,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-02-05 02:53:13,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-02-05 02:53:13,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-02-05 02:53:13,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-02-05 02:53:13,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-02-05 02:53:13,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-02-05 02:53:13,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-02-05 02:53:13,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-02-05 02:53:13,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-02-05 02:53:13,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-02-05 02:53:13,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-02-05 02:53:13,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-02-05 02:53:13,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-02-05 02:53:13,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-02-05 02:53:13,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-02-05 02:53:13,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-02-05 02:53:13,232 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02363468487136436); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.030218062696528263, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02363468487136436); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012083333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.049166666666666664, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-05 02:53:13,232 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-02-05 02:53:13,232 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.015554557538954452, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02363468487136436); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.030218062696528263, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02363468487136436); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012083333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.049166666666666664, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-02-05 02:53:13,448 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 324 ms to try 5000 different iterations.  Found a solution that moves 1020 regions; Going from a computed imbalance of 0.015554557538954452 to a new imbalance of 7.994262318982839E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007332959212304937); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006496135430631728); MoveCostFunction : (multiplier=7.0, imbalance=0.11333333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007332959212304937); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00475); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-05 02:53:13,452 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv70994355:79 , srv1236284563:83 , srv1272681439:83 , srv1382153121:83 , srv1608877635:83 , srv22936862:83 , srv373399525:83 , srv564510362:83 , srv582038221:83 , srv613345306:83 , srv617430806:83 , srv628096652:83 , srv721167113:83 , srv803764986:83 , srv1061184343:84 , srv1099859862:84 , srv1182953725:84 , srv1264468192:84 , srv1265062165:84 , srv1311579464:84 , srv136055579:84 , srv1378835569:84 , srv1447956787:84 , srv1467549296:84 , srv1680493983:84 , srv2045399544:84 , srv789190436:84 , srv871907884:84 , srv1009236336:85 , srv1054393396:85 , srv1177891322:85 , srv1328471885:85 , srv1378768350:85 , srv1618029376:85 , srv1797593631:85 , srv180334465:85 , srv205125390:85 , srv2497583:85 , srv255968183:85 , srv298649200:85 , srv365372721:85 , srv477634312:85 , srv505933894:85 , srv506563563:85 , srv573552943:85 , srv1042480651:86 , srv1118336637:86 , srv1285009451:86 , srv1290108101:86 , srv133609377:86 , srv1439047171:86 , srv1458831965:86 , srv1473415808:86 , srv1563770918:86 , srv1569114821:86 , srv1615495159:86 , srv1674835218:86 , srv1685284687:86 , srv1780015094:86 , srv1848776723:86 , srv1872019917:86 , srv2011300067:86 , srv2016516989:86 , srv257125741:86 , srv280375379:86 , srv363734574:86 , srv480994044:86 , srv530010161:86 , srv55306089:86 , srv646767629:86 , srv786990502:86 , srv839310002:86 , srv943737694:86 , srv1379637598:87 , srv1389869859:87 , srv1404176955:87 , srv1461906324:87 , srv1550517029:87 , srv1595847305:87 , srv1645009488:87 , srv1853426961:87 , srv1857346222:87 , srv1895072614:87 , srv1950909052:87 , srv22485517:87 , srv315098169:87 , srv355963012:87 , srv404588942:87 , srv594371146:87 , srv1214843210:88 , srv1273519014:88 , srv1291080411:88 , srv199796600:88 , srv305702596:88 , srv322580374:88 , srv773628487:88 , srv997735341:88 , srv301456363:89 , srv649478036:89 , srv242511437:539 }
2024-02-05 02:53:13,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-05 02:53:13,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1273519014=13, srv205125390=57, srv839310002=96, srv1473415808=32, srv1674835218=41, srv2045399544=56, srv1236284563=9, srv1272681439=12, srv786990502=93, srv1569114821=35, srv280375379=64, srv1264468192=10, srv505933894=77, srv2497583=61, srv2016516989=55, srv1378768350=21, srv257125741=63, srv1061184343=3, srv373399525=73, srv1042480651=1, srv1378835569=22, srv997735341=99, srv613345306=85, srv1857346222=49, srv355963012=70, srv1118336637=5, srv1439047171=27, srv322580374=69, srv365372721=72, srv1404176955=26, srv1950909052=52, srv1608877635=37, srv180334465=46, srv871907884=97, srv2011300067=54, srv594371146=84, srv1099859862=4, srv315098169=68, srv363734574=71, srv477634312=75, srv1895072614=51, srv22936862=59, srv22485517=58, srv1615495159=38, srv582038221=83, srv480994044=76, srv943737694=98, srv255968183=62, srv1054393396=2, srv1680493983=42, srv789190436=94, srv404588942=74, srv70994355=90, srv1780015094=44, srv649478036=89, srv1291080411=16, srv1550517029=33, srv506563563=78, srv1645009488=40, srv133609377=19, srv1182953725=7, srv628096652=87, srv1389869859=25, srv1618029376=39, srv55306089=80, srv1685284687=43, srv721167113=91, srv1177891322=6, srv1379637598=23, srv301456363=66, srv136055579=20, srv1872019917=50, srv242511437=60, srv1595847305=36, srv1009236336=0, srv305702596=67, srv1328471885=18, srv1461906324=30, srv1853426961=48, srv773628487=92, srv1214843210=8, srv1467549296=31, srv199796600=53, srv1797593631=45, srv646767629=88, srv564510362=81, srv1458831965=29, srv1563770918=34, srv530010161=79, srv1311579464=17, srv298649200=65, srv1265062165=11, srv1447956787=28, srv1285009451=14, srv1290108101=15, srv617430806=86, srv1382153121=24, srv1848776723=47, srv573552943=82, srv803764986=95} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-05 02:53:13,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-05 02:53:13,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-05 02:53:13,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-05 02:53:13,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-05 02:53:13,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-05 02:53:13,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-05 02:53:13,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-05 02:53:13,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-05 02:53:13,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-05 02:53:13,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-05 02:53:13,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-05 02:53:13,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-05 02:53:13,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-05 02:53:13,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-05 02:53:13,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-05 02:53:13,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-05 02:53:13,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-05 02:53:13,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-05 02:53:13,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-05 02:53:13,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-05 02:53:13,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-05 02:53:13,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-05 02:53:13,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-05 02:53:13,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-05 02:53:13,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-05 02:53:13,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-05 02:53:13,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-05 02:53:13,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-05 02:53:13,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-05 02:53:13,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-05 02:53:13,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-02-05 02:53:13,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-02-05 02:53:13,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-02-05 02:53:13,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-02-05 02:53:13,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-02-05 02:53:13,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-02-05 02:53:13,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-02-05 02:53:13,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-02-05 02:53:13,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-02-05 02:53:13,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-02-05 02:53:13,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-02-05 02:53:13,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-02-05 02:53:13,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-02-05 02:53:13,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-02-05 02:53:13,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-02-05 02:53:13,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-02-05 02:53:13,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-02-05 02:53:13,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-02-05 02:53:13,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-02-05 02:53:13,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-02-05 02:53:13,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-02-05 02:53:13,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-02-05 02:53:13,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-02-05 02:53:13,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-02-05 02:53:13,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-02-05 02:53:13,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-02-05 02:53:13,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-02-05 02:53:13,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-02-05 02:53:13,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-02-05 02:53:13,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-02-05 02:53:13,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-02-05 02:53:13,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-02-05 02:53:13,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-02-05 02:53:13,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-02-05 02:53:13,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-02-05 02:53:13,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-02-05 02:53:13,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-02-05 02:53:13,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-02-05 02:53:13,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-02-05 02:53:13,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-02-05 02:53:13,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-02-05 02:53:13,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-02-05 02:53:13,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-02-05 02:53:13,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-02-05 02:53:13,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-02-05 02:53:13,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-02-05 02:53:13,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-02-05 02:53:13,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-02-05 02:53:13,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-02-05 02:53:13,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-02-05 02:53:13,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-02-05 02:53:13,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-02-05 02:53:13,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-02-05 02:53:13,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-02-05 02:53:13,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-02-05 02:53:13,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-02-05 02:53:13,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-02-05 02:53:13,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-02-05 02:53:13,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-02-05 02:53:13,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-02-05 02:53:13,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-02-05 02:53:13,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-02-05 02:53:13,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-02-05 02:53:13,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-02-05 02:53:13,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-02-05 02:53:13,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-02-05 02:53:13,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-02-05 02:53:13,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-02-05 02:53:13,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-02-05 02:53:13,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-02-05 02:53:13,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-02-05 02:53:13,602 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007332959212304937); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006496135430631728); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007332959212304937); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00475); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-05 02:53:13,602 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-02-05 02:53:13,603 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=7.922840392451393E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007332959212304937); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006496135430631728); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007332959212304937); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00475); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-02-05 02:53:13,982 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 486 ms to try 5000 different iterations.  Found a solution that moves 299 regions; Going from a computed imbalance of 7.922840392451393E-4 to a new imbalance of 4.2225822484676466E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.868392354571063E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.03322222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.868392354571063E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-05 02:53:13,983 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv649478036:82 , srv1272681439:83 , srv1285009451:83 , srv1550517029:83 , srv180334465:83 , srv199796600:83 , srv646767629:83 , srv1182953725:84 , srv1265062165:84 , srv1273519014:84 , srv1291080411:84 , srv1328471885:84 , srv1439047171:84 , srv1447956787:84 , srv1458831965:84 , srv1461906324:84 , srv1569114821:84 , srv1608877635:84 , srv1645009488:84 , srv1797593631:84 , srv2016516989:84 , srv2497583:84 , srv301456363:84 , srv322580374:84 , srv355963012:84 , srv365372721:84 , srv480994044:84 , srv505933894:84 , srv564510362:84 , srv594371146:84 , srv773628487:84 , srv839310002:84 , srv943737694:84 , srv1009236336:85 , srv1042480651:85 , srv1054393396:85 , srv1061184343:85 , srv1099859862:85 , srv1118336637:85 , srv1177891322:85 , srv1214843210:85 , srv1236284563:85 , srv1264468192:85 , srv1290108101:85 , srv1311579464:85 , srv133609377:85 , srv1378768350:85 , srv1378835569:85 , srv1379637598:85 , srv1382153121:85 , srv1389869859:85 , srv1404176955:85 , srv1473415808:85 , srv1563770918:85 , srv1595847305:85 , srv1615495159:85 , srv1674835218:85 , srv1680493983:85 , srv1685284687:85 , srv1780015094:85 , srv1848776723:85 , srv1853426961:85 , srv1857346222:85 , srv1872019917:85 , srv1895072614:85 , srv1950909052:85 , srv2011300067:85 , srv2045399544:85 , srv205125390:85 , srv22485517:85 , srv255968183:85 , srv257125741:85 , srv280375379:85 , srv298649200:85 , srv305702596:85 , srv315098169:85 , srv363734574:85 , srv373399525:85 , srv404588942:85 , srv477634312:85 , srv506563563:85 , srv530010161:85 , srv55306089:85 , srv573552943:85 , srv613345306:85 , srv617430806:85 , srv721167113:85 , srv786990502:85 , srv789190436:85 , srv803764986:85 , srv871907884:85 , srv997735341:85 , srv136055579:86 , srv1467549296:86 , srv1618029376:86 , srv22936862:86 , srv582038221:86 , srv628096652:86 , srv70994355:86 , srv242511437:619 }
2024-02-05 02:53:13,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-05 02:53:13,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1273519014=13, srv205125390=57, srv839310002=96, srv1473415808=32, srv1674835218=41, srv2045399544=56, srv1236284563=9, srv1272681439=12, srv786990502=93, srv1569114821=35, srv280375379=64, srv1264468192=10, srv505933894=77, srv2497583=61, srv2016516989=55, srv1378768350=21, srv257125741=63, srv1061184343=3, srv373399525=73, srv1042480651=1, srv1378835569=22, srv997735341=99, srv613345306=85, srv1857346222=49, srv355963012=70, srv1118336637=5, srv1439047171=27, srv322580374=69, srv365372721=72, srv1404176955=26, srv1950909052=52, srv1608877635=37, srv180334465=46, srv871907884=97, srv2011300067=54, srv594371146=84, srv1099859862=4, srv315098169=68, srv363734574=71, srv477634312=75, srv1895072614=51, srv22936862=59, srv22485517=58, srv1615495159=38, srv582038221=83, srv480994044=76, srv943737694=98, srv255968183=62, srv1054393396=2, srv1680493983=42, srv789190436=94, srv404588942=74, srv70994355=90, srv1780015094=44, srv649478036=89, srv1291080411=16, srv1550517029=33, srv506563563=78, srv1645009488=40, srv133609377=19, srv1182953725=7, srv628096652=87, srv1389869859=25, srv1618029376=39, srv55306089=80, srv1685284687=43, srv721167113=91, srv1177891322=6, srv1379637598=23, srv301456363=66, srv136055579=20, srv1872019917=50, srv242511437=60, srv1595847305=36, srv1009236336=0, srv305702596=67, srv1328471885=18, srv1461906324=30, srv1853426961=48, srv773628487=92, srv1214843210=8, srv1467549296=31, srv199796600=53, srv1797593631=45, srv646767629=88, srv564510362=81, srv1458831965=29, srv1563770918=34, srv530010161=79, srv1311579464=17, srv298649200=65, srv1265062165=11, srv1447956787=28, srv1285009451=14, srv1290108101=15, srv617430806=86, srv1382153121=24, srv1848776723=47, srv573552943=82, srv803764986=95} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-05 02:53:14,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-05 02:53:14,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-05 02:53:14,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-05 02:53:14,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-05 02:53:14,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-05 02:53:14,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-05 02:53:14,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-05 02:53:14,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-05 02:53:14,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-05 02:53:14,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-05 02:53:14,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-05 02:53:14,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-05 02:53:14,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-05 02:53:14,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-05 02:53:14,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-05 02:53:14,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-05 02:53:14,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-05 02:53:14,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-05 02:53:14,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-05 02:53:14,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-05 02:53:14,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-05 02:53:14,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-05 02:53:14,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-05 02:53:14,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-05 02:53:14,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-05 02:53:14,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-05 02:53:14,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-05 02:53:14,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-05 02:53:14,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-05 02:53:14,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-05 02:53:14,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-02-05 02:53:14,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-02-05 02:53:14,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-02-05 02:53:14,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-02-05 02:53:14,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-02-05 02:53:14,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-02-05 02:53:14,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-02-05 02:53:14,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-02-05 02:53:14,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-02-05 02:53:14,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-02-05 02:53:14,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-02-05 02:53:14,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-02-05 02:53:14,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-02-05 02:53:14,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-02-05 02:53:14,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-02-05 02:53:14,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-02-05 02:53:14,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-02-05 02:53:14,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-02-05 02:53:14,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-02-05 02:53:14,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-02-05 02:53:14,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-02-05 02:53:14,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-02-05 02:53:14,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-02-05 02:53:14,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-02-05 02:53:14,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-02-05 02:53:14,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-02-05 02:53:14,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-02-05 02:53:14,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-02-05 02:53:14,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-02-05 02:53:14,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-02-05 02:53:14,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-02-05 02:53:14,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-02-05 02:53:14,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-02-05 02:53:14,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-02-05 02:53:14,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-02-05 02:53:14,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-02-05 02:53:14,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-02-05 02:53:14,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-02-05 02:53:14,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-02-05 02:53:14,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-02-05 02:53:14,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-02-05 02:53:14,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-02-05 02:53:14,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-02-05 02:53:14,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-02-05 02:53:14,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-02-05 02:53:14,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-02-05 02:53:14,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-02-05 02:53:14,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-02-05 02:53:14,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-02-05 02:53:14,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-02-05 02:53:14,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-02-05 02:53:14,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-02-05 02:53:14,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-02-05 02:53:14,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-02-05 02:53:14,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-02-05 02:53:14,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-02-05 02:53:14,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-02-05 02:53:14,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-02-05 02:53:14,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-02-05 02:53:14,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-02-05 02:53:14,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-02-05 02:53:14,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-02-05 02:53:14,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-02-05 02:53:14,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-02-05 02:53:14,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-02-05 02:53:14,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-02-05 02:53:14,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-02-05 02:53:14,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-02-05 02:53:14,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-02-05 02:53:14,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-02-05 02:53:14,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-02-05 02:53:14,163 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.868392354571063E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.868392354571063E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-05 02:53:14,163 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-02-05 02:53:14,164 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=4.013217973635273E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.868392354571063E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.868392354571063E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-02-05 02:53:14,419 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 384 ms to try 5000 different iterations.  Found a solution that moves 27 regions; Going from a computed imbalance of 4.013217973635273E-5 to a new imbalance of 7.262297337147979E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0015713484026367722); MoveCostFunction : (multiplier=7.0, imbalance=0.003); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-05 02:53:14,419 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv136055579:83 , srv315098169:83 , srv373399525:83 , srv530010161:83 , srv55306089:83 , srv70994355:83 , srv1009236336:84 , srv1042480651:84 , srv1054393396:84 , srv1061184343:84 , srv1099859862:84 , srv1118336637:84 , srv1177891322:84 , srv1182953725:84 , srv1214843210:84 , srv1236284563:84 , srv1264468192:84 , srv1265062165:84 , srv1290108101:84 , srv1291080411:84 , srv1311579464:84 , srv1328471885:84 , srv133609377:84 , srv1378768350:84 , srv1378835569:84 , srv1379637598:84 , srv1382153121:84 , srv1389869859:84 , srv1404176955:84 , srv1439047171:84 , srv1447956787:84 , srv1458831965:84 , srv1461906324:84 , srv1467549296:84 , srv1473415808:84 , srv1550517029:84 , srv1563770918:84 , srv1569114821:84 , srv1595847305:84 , srv1615495159:84 , srv1618029376:84 , srv1674835218:84 , srv1680493983:84 , srv1685284687:84 , srv1780015094:84 , srv1797593631:84 , srv180334465:84 , srv1848776723:84 , srv1853426961:84 , srv1857346222:84 , srv1872019917:84 , srv1895072614:84 , srv1950909052:84 , srv199796600:84 , srv2011300067:84 , srv2016516989:84 , srv2045399544:84 , srv205125390:84 , srv22485517:84 , srv22936862:84 , srv2497583:84 , srv255968183:84 , srv257125741:84 , srv280375379:84 , srv298649200:84 , srv301456363:84 , srv305702596:84 , srv322580374:84 , srv355963012:84 , srv363734574:84 , srv365372721:84 , srv404588942:84 , srv477634312:84 , srv480994044:84 , srv505933894:84 , srv506563563:84 , srv564510362:84 , srv573552943:84 , srv582038221:84 , srv594371146:84 , srv613345306:84 , srv617430806:84 , srv628096652:84 , srv649478036:84 , srv721167113:84 , srv773628487:84 , srv786990502:84 , srv789190436:84 , srv803764986:84 , srv839310002:84 , srv871907884:84 , srv943737694:84 , srv997735341:84 , srv1272681439:85 , srv1273519014:85 , srv1285009451:85 , srv1608877635:85 , srv1645009488:85 , srv646767629:85 , srv242511437:684 }
2024-02-05 02:53:14,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-05 02:53:14,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1273519014=13, srv205125390=57, srv839310002=96, srv1473415808=32, srv1674835218=41, srv2045399544=56, srv1236284563=9, srv1272681439=12, srv786990502=93, srv1569114821=35, srv280375379=64, srv1264468192=10, srv505933894=77, srv2497583=61, srv2016516989=55, srv1378768350=21, srv257125741=63, srv1061184343=3, srv373399525=73, srv1042480651=1, srv1378835569=22, srv997735341=99, srv613345306=85, srv1857346222=49, srv355963012=70, srv1118336637=5, srv1439047171=27, srv322580374=69, srv365372721=72, srv1404176955=26, srv1950909052=52, srv1608877635=37, srv180334465=46, srv871907884=97, srv2011300067=54, srv594371146=84, srv1099859862=4, srv315098169=68, srv363734574=71, srv477634312=75, srv1895072614=51, srv22936862=59, srv22485517=58, srv1615495159=38, srv582038221=83, srv480994044=76, srv943737694=98, srv255968183=62, srv1054393396=2, srv1680493983=42, srv789190436=94, srv404588942=74, srv70994355=90, srv1780015094=44, srv649478036=89, srv1291080411=16, srv1550517029=33, srv506563563=78, srv1645009488=40, srv133609377=19, srv1182953725=7, srv628096652=87, srv1389869859=25, srv1618029376=39, srv55306089=80, srv1685284687=43, srv721167113=91, srv1177891322=6, srv1379637598=23, srv301456363=66, srv136055579=20, srv1872019917=50, srv242511437=60, srv1595847305=36, srv1009236336=0, srv305702596=67, srv1328471885=18, srv1461906324=30, srv1853426961=48, srv773628487=92, srv1214843210=8, srv1467549296=31, srv199796600=53, srv1797593631=45, srv646767629=88, srv564510362=81, srv1458831965=29, srv1563770918=34, srv530010161=79, srv1311579464=17, srv298649200=65, srv1265062165=11, srv1447956787=28, srv1285009451=14, srv1290108101=15, srv617430806=86, srv1382153121=24, srv1848776723=47, srv573552943=82, srv803764986=95} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-05 02:53:14,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-05 02:53:14,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-05 02:53:14,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-05 02:53:14,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-05 02:53:14,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-05 02:53:14,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-05 02:53:14,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-05 02:53:14,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-05 02:53:14,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-05 02:53:14,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-05 02:53:14,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-05 02:53:14,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-05 02:53:14,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-05 02:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-05 02:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-05 02:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-05 02:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-05 02:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-05 02:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-05 02:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-05 02:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-05 02:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-05 02:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-05 02:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-05 02:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-05 02:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-05 02:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-05 02:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-05 02:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-05 02:53:14,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-05 02:53:14,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-02-05 02:53:14,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-02-05 02:53:14,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-02-05 02:53:14,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-02-05 02:53:14,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-02-05 02:53:14,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-02-05 02:53:14,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-02-05 02:53:14,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-02-05 02:53:14,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-02-05 02:53:14,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-02-05 02:53:14,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-02-05 02:53:14,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-02-05 02:53:14,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-02-05 02:53:14,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-02-05 02:53:14,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-02-05 02:53:14,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-02-05 02:53:14,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-02-05 02:53:14,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-02-05 02:53:14,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-02-05 02:53:14,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-02-05 02:53:14,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-02-05 02:53:14,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-02-05 02:53:14,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-02-05 02:53:14,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-02-05 02:53:14,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-02-05 02:53:14,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-02-05 02:53:14,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-02-05 02:53:14,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-02-05 02:53:14,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-02-05 02:53:14,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-02-05 02:53:14,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-02-05 02:53:14,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-02-05 02:53:14,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-02-05 02:53:14,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-02-05 02:53:14,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-02-05 02:53:14,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-02-05 02:53:14,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-02-05 02:53:14,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-02-05 02:53:14,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-02-05 02:53:14,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-02-05 02:53:14,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-02-05 02:53:14,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-02-05 02:53:14,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-02-05 02:53:14,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-02-05 02:53:14,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-02-05 02:53:14,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-02-05 02:53:14,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-02-05 02:53:14,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-02-05 02:53:14,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-02-05 02:53:14,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-02-05 02:53:14,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-02-05 02:53:14,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-02-05 02:53:14,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-02-05 02:53:14,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-02-05 02:53:14,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-02-05 02:53:14,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-02-05 02:53:14,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-02-05 02:53:14,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-02-05 02:53:14,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-02-05 02:53:14,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-02-05 02:53:14,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-02-05 02:53:14,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-02-05 02:53:14,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-02-05 02:53:14,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-02-05 02:53:14,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-02-05 02:53:14,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-02-05 02:53:14,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-02-05 02:53:14,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-02-05 02:53:14,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-02-05 02:53:14,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-02-05 02:53:14,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-02-05 02:53:14,595 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=7.073239296329447E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0015713484026367722); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-05 02:53:14,596 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv136055579:83 , srv315098169:83 , srv373399525:83 , srv530010161:83 , srv55306089:83 , srv70994355:83 , srv1009236336:84 , srv1042480651:84 , srv1054393396:84 , srv1061184343:84 , srv1099859862:84 , srv1118336637:84 , srv1177891322:84 , srv1182953725:84 , srv1214843210:84 , srv1236284563:84 , srv1264468192:84 , srv1265062165:84 , srv1290108101:84 , srv1291080411:84 , srv1311579464:84 , srv1328471885:84 , srv133609377:84 , srv1378768350:84 , srv1378835569:84 , srv1379637598:84 , srv1382153121:84 , srv1389869859:84 , srv1404176955:84 , srv1439047171:84 , srv1447956787:84 , srv1458831965:84 , srv1461906324:84 , srv1467549296:84 , srv1473415808:84 , srv1550517029:84 , srv1563770918:84 , srv1569114821:84 , srv1595847305:84 , srv1615495159:84 , srv1618029376:84 , srv1674835218:84 , srv1680493983:84 , srv1685284687:84 , srv1780015094:84 , srv1797593631:84 , srv180334465:84 , srv1848776723:84 , srv1853426961:84 , srv1857346222:84 , srv1872019917:84 , srv1895072614:84 , srv1950909052:84 , srv199796600:84 , srv2011300067:84 , srv2016516989:84 , srv2045399544:84 , srv205125390:84 , srv22485517:84 , srv22936862:84 , srv2497583:84 , srv255968183:84 , srv257125741:84 , srv280375379:84 , srv298649200:84 , srv301456363:84 , srv305702596:84 , srv322580374:84 , srv355963012:84 , srv363734574:84 , srv365372721:84 , srv404588942:84 , srv477634312:84 , srv480994044:84 , srv505933894:84 , srv506563563:84 , srv564510362:84 , srv573552943:84 , srv582038221:84 , srv594371146:84 , srv613345306:84 , srv617430806:84 , srv628096652:84 , srv649478036:84 , srv721167113:84 , srv773628487:84 , srv786990502:84 , srv789190436:84 , srv803764986:84 , srv839310002:84 , srv871907884:84 , srv943737694:84 , srv997735341:84 , srv1272681439:85 , srv1273519014:85 , srv1285009451:85 , srv1608877635:85 , srv1645009488:85 , srv646767629:85 , srv242511437:684 }
2024-02-05 02:53:14,633 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=284 (was 284), ProcessCount=169 (was 169), AvailableMemoryMB=2305 (was 3214)
2024-02-05 02:53:14,640 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=284, ProcessCount=169, AvailableMemoryMB=2305
2024-02-05 02:53:14,641 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-02-05 02:53:14,641 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2024-02-05 02:53:14,653 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1047749486:84 , srv1175584893:84 , srv1381453444:84 , srv1407163360:84 , srv153759770:84 , srv1586574442:84 , srv1598585084:84 , srv1816822446:84 , srv1821437194:84 , srv1839795600:84 , srv1848533882:84 , srv1876937253:84 , srv1976508912:84 , srv1977927361:84 , srv2030878477:84 , srv2129799449:84 , srv2136094492:84 , srv382455780:84 , srv406895321:84 , srv427574614:84 , srv469376491:84 , srv647376416:84 , srv66669719:84 , srv71009300:84 , srv809694827:84 , srv89615618:84 , srv935673554:84 , srv941718547:84 , srv96879207:84 , srv1421331405:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2024-02-05 02:53:14,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-05 02:53:14,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv935673554=27, srv1047749486=0, srv1175584893=1, srv1848533882=11, srv1586574442=6, srv1821437194=9, srv941718547=28, srv89615618=26, srv1977927361=14, srv1598585084=7, srv427574614=20, srv1381453444=2, srv96879207=29, srv1407163360=3, srv469376491=21, srv1876937253=12, srv2129799449=16, srv1839795600=10, srv2136094492=17, srv647376416=22, srv809694827=25, srv2030878477=15, srv382455780=18, srv1421331405=4, srv1976508912=13, srv406895321=19, srv66669719=23, srv153759770=5, srv1816822446=8, srv71009300=24} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-05 02:53:14,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-05 02:53:14,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-05 02:53:14,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-05 02:53:14,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-05 02:53:14,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-05 02:53:14,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-05 02:53:14,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-05 02:53:14,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-05 02:53:14,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-05 02:53:14,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-05 02:53:14,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-05 02:53:14,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-05 02:53:14,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-05 02:53:14,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-05 02:53:14,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-05 02:53:14,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-05 02:53:14,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-05 02:53:14,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-05 02:53:14,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-05 02:53:14,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-05 02:53:14,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-05 02:53:14,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-05 02:53:14,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-05 02:53:14,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-05 02:53:14,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-05 02:53:14,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-05 02:53:14,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-05 02:53:14,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-05 02:53:14,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-05 02:53:14,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-05 02:53:14,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-02-05 02:53:14,694 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1335688053709867, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-05 02:53:14,694 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912613914208581, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1335688053709867, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2024-02-05 02:55:14,674 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 7352735 different iterations.  Found a solution that moves 1845 regions; Going from a computed imbalance of 0.9912613914208581 to a new imbalance of 4.306322040866546E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6833333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-05 02:55:14,680 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1047749486:90 , srv1175584893:90 , srv1381453444:90 , srv1407163360:90 , srv1421331405:90 , srv153759770:90 , srv1586574442:90 , srv1598585084:90 , srv1816822446:90 , srv1821437194:90 , srv1839795600:90 , srv1848533882:90 , srv1876937253:90 , srv1976508912:90 , srv1977927361:90 , srv2030878477:90 , srv2129799449:90 , srv2136094492:90 , srv382455780:90 , srv406895321:90 , srv427574614:90 , srv469376491:90 , srv647376416:90 , srv66669719:90 , srv71009300:90 , srv809694827:90 , srv89615618:90 , srv935673554:90 , srv941718547:90 , srv96879207:90 }
2024-02-05 02:55:14,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-05 02:55:14,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv935673554=27, srv1047749486=0, srv1175584893=1, srv1848533882=11, srv1586574442=6, srv1821437194=9, srv941718547=28, srv89615618=26, srv1977927361=14, srv1598585084=7, srv427574614=20, srv1381453444=2, srv96879207=29, srv1407163360=3, srv469376491=21, srv1876937253=12, srv2129799449=16, srv1839795600=10, srv2136094492=17, srv647376416=22, srv809694827=25, srv2030878477=15, srv382455780=18, srv1421331405=4, srv1976508912=13, srv406895321=19, srv66669719=23, srv153759770=5, srv1816822446=8, srv71009300=24} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-05 02:55:14,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-05 02:55:14,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-05 02:55:14,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-05 02:55:14,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-05 02:55:14,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-05 02:55:14,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-05 02:55:14,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-05 02:55:14,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-05 02:55:14,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-05 02:55:14,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-05 02:55:14,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-05 02:55:14,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-05 02:55:14,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-05 02:55:14,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-05 02:55:14,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-05 02:55:14,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-05 02:55:14,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-05 02:55:14,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-05 02:55:14,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-05 02:55:14,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-05 02:55:14,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-05 02:55:14,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-05 02:55:14,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-05 02:55:14,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-05 02:55:14,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-05 02:55:14,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-05 02:55:14,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-05 02:55:14,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-05 02:55:14,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-05 02:55:14,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-05 02:55:14,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-02-05 02:55:14,697 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-05 02:55:14,697 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1047749486:90 , srv1175584893:90 , srv1381453444:90 , srv1407163360:90 , srv1421331405:90 , srv153759770:90 , srv1586574442:90 , srv1598585084:90 , srv1816822446:90 , srv1821437194:90 , srv1839795600:90 , srv1848533882:90 , srv1876937253:90 , srv1976508912:90 , srv1977927361:90 , srv2030878477:90 , srv2129799449:90 , srv2136094492:90 , srv382455780:90 , srv406895321:90 , srv427574614:90 , srv469376491:90 , srv647376416:90 , srv66669719:90 , srv71009300:90 , srv809694827:90 , srv89615618:90 , srv935673554:90 , srv941718547:90 , srv96879207:90 }
2024-02-05 02:55:14,709 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=173 (was 284), ProcessCount=168 (was 169), AvailableMemoryMB=2603 (was 2305) - AvailableMemoryMB LEAK? -