2023-12-11 12:53:13,345 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-12-11 12:53:13,657 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-12-11 12:53:13,776 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-12-11 12:53:13,842 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-12-11 12:53:13,849 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-12-11 12:53:13,866 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=281, ProcessCount=171, AvailableMemoryMB=6651
2023-12-11 12:53:13,868 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-12-11 12:53:13,869 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-12-11 12:53:13,990 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv100266818:84 , srv1006523404:84 , srv1042841076:84 , srv1047223016:84 , srv1073625498:84 , srv1094764011:84 , srv1100502586:84 , srv1104185981:84 , srv1154642200:84 , srv1161486341:84 , srv1166756303:84 , srv1200470085:84 , srv1201261872:84 , srv1203248938:84 , srv1237939703:84 , srv1276792465:84 , srv1324009065:84 , srv1349073112:84 , srv140007689:84 , srv140238226:84 , srv1416993589:84 , srv1423702748:84 , srv1490146528:84 , srv150427401:84 , srv1530204620:84 , srv1534580307:84 , srv1563032344:84 , srv1602780958:84 , srv1640211861:84 , srv1653669488:84 , srv1655213260:84 , srv1678332032:84 , srv1678684432:84 , srv1683166178:84 , srv1700579752:84 , srv1707785418:84 , srv1727016423:84 , srv1727362576:84 , srv1759345384:84 , srv1762608951:84 , srv1778331528:84 , srv1804661470:84 , srv1812725747:84 , srv1837346990:84 , srv1862016463:84 , srv1880368562:84 , srv1900763356:84 , srv1902376498:84 , srv1918591489:84 , srv1935351078:84 , srv1950613792:84 , srv1993535269:84 , srv2004344448:84 , srv2025092740:84 , srv2069423735:84 , srv2085850088:84 , srv2088789750:84 , srv2089021771:84 , srv2097138400:84 , srv2118988742:84 , srv2130726093:84 , srv219372408:84 , srv220159090:84 , srv224816288:84 , srv264029095:84 , srv28806147:84 , srv295614185:84 , srv312173128:84 , srv326785800:84 , srv360291700:84 , srv382499618:84 , srv404579205:84 , srv414301021:84 , srv451866007:84 , srv476293428:84 , srv517686089:84 , srv531285800:84 , srv536650559:84 , srv658625345:84 , srv672406718:84 , srv688111423:84 , srv691274278:84 , srv728915235:84 , srv748687804:84 , srv811506009:84 , srv815450682:84 , srv823760974:84 , srv841229950:84 , srv859214431:84 , srv891799089:84 , srv899758779:84 , srv913469947:84 , srv918300185:84 , srv940916546:84 , srv945053995:84 , srv945226867:84 , srv968203370:84 , srv983528869:84 , srv987748917:84 , srv1036552000:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-12-11 12:53:13,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-11 12:53:14,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1653669488=30, srv1862016463=45, srv2118988742=60, srv1837346990=44, srv1166756303=11, srv476293428=75, srv326785800=69, srv1950613792=51, srv859214431=89, srv688111423=81, srv672406718=80, srv1700579752=35, srv1490146528=23, srv2085850088=56, srv728915235=83, srv312173128=68, srv264029095=65, srv219372408=62, srv531285800=77, srv1094764011=6, srv658625345=79, srv295614185=67, srv1683166178=34, srv2088789750=57, srv1237939703=15, srv815450682=86, srv1416993589=21, srv1880368562=46, srv1900763356=47, srv1324009065=17, srv1006523404=1, srv2130726093=61, srv451866007=74, srv1762608951=40, srv517686089=76, srv691274278=82, srv811506009=85, srv918300185=93, srv1655213260=31, srv1200470085=12, srv1161486341=10, srv404579205=72, srv1707785418=36, srv1640211861=29, srv987748917=99, srv945053995=95, srv1201261872=13, srv150427401=24, srv945226867=96, srv913469947=92, srv1073625498=5, srv1678332032=32, srv1423702748=22, srv968203370=97, srv414301021=73, srv1918591489=49, srv940916546=94, srv1778331528=41, srv1727362576=38, srv1276792465=16, srv140007689=19, srv1812725747=43, srv28806147=66, srv983528869=98, srv1563032344=27, srv360291700=70, srv823760974=87, srv2097138400=59, srv1902376498=48, srv100266818=0, srv1727016423=37, srv748687804=84, srv841229950=88, srv1203248938=14, srv140238226=20, srv1534580307=26, srv1993535269=52, srv1154642200=9, srv891799089=90, srv1602780958=28, srv1349073112=18, srv1047223016=4, srv1804661470=42, srv1530204620=25, srv382499618=71, srv1104185981=8, srv220159090=63, srv536650559=78, srv2025092740=54, srv2004344448=53, srv2069423735=55, srv1935351078=50, srv224816288=64, srv1100502586=7, srv2089021771=58, srv1678684432=33, srv1036552000=2, srv1042841076=3, srv899758779=91, srv1759345384=39} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-11 12:53:14,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-11 12:53:14,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-11 12:53:14,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-11 12:53:14,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-11 12:53:14,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-11 12:53:14,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-11 12:53:14,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-11 12:53:14,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-11 12:53:14,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-11 12:53:14,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-11 12:53:14,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-11 12:53:14,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-11 12:53:14,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-11 12:53:14,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-11 12:53:14,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-11 12:53:14,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-11 12:53:14,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-11 12:53:14,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-11 12:53:14,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-11 12:53:14,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-11 12:53:14,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-11 12:53:14,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-11 12:53:14,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-11 12:53:14,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-11 12:53:14,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-11 12:53:14,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-11 12:53:14,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-11 12:53:14,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-11 12:53:14,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-11 12:53:14,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-11 12:53:14,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-11 12:53:14,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-11 12:53:14,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-11 12:53:14,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-11 12:53:14,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-11 12:53:14,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-11 12:53:14,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-11 12:53:14,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-11 12:53:14,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-11 12:53:14,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-11 12:53:14,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-11 12:53:14,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-11 12:53:14,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-11 12:53:14,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-11 12:53:14,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-11 12:53:14,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-11 12:53:14,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-11 12:53:14,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-11 12:53:14,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-11 12:53:14,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-11 12:53:14,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-11 12:53:14,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-11 12:53:14,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-11 12:53:14,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-11 12:53:14,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-11 12:53:14,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-11 12:53:14,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-11 12:53:14,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-11 12:53:14,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-11 12:53:14,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-11 12:53:14,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-11 12:53:14,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-11 12:53:14,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-11 12:53:14,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-11 12:53:14,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-11 12:53:14,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-11 12:53:14,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-11 12:53:14,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-11 12:53:14,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-11 12:53:14,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-11 12:53:14,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-11 12:53:14,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-11 12:53:14,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-11 12:53:14,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-11 12:53:14,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-11 12:53:14,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-11 12:53:14,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-11 12:53:14,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-11 12:53:14,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-11 12:53:14,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-11 12:53:14,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-11 12:53:14,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-11 12:53:14,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-11 12:53:14,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-11 12:53:14,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-11 12:53:14,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-11 12:53:14,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-11 12:53:14,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-11 12:53:14,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-11 12:53:14,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-11 12:53:14,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-11 12:53:14,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-11 12:53:14,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-11 12:53:14,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-11 12:53:14,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-11 12:53:14,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-11 12:53:14,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-11 12:53:14,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-11 12:53:14,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-11 12:53:14,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-11 12:53:14,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-11 12:53:14,292 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-11 12:53:14,292 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-12-11 12:53:14,293 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-12-11 12:53:14,719 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 583 ms to try 5000 different iterations.  Found a solution that moves 4696 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18282302146327511. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04231073479729663, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05728813503042098, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5217777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04231073479729663, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17825, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24275, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-11 12:53:14,730 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1640211861:81 , srv1237939703:82 , srv1778331528:82 , srv264029095:82 , srv913469947:82 , srv1006523404:83 , srv1100502586:83 , srv1416993589:83 , srv1678332032:83 , srv1837346990:83 , srv1900763356:83 , srv312173128:83 , srv531285800:83 , srv859214431:83 , srv1042841076:84 , srv1094764011:84 , srv1200470085:84 , srv1530204620:84 , srv1602780958:84 , srv1678684432:84 , srv1700579752:84 , srv1727016423:84 , srv1759345384:84 , srv1880368562:84 , srv2088789750:84 , srv295614185:84 , srv476293428:84 , srv688111423:84 , srv815450682:84 , srv940916546:84 , srv983528869:84 , srv1104185981:85 , srv1154642200:85 , srv1201261872:85 , srv1423702748:85 , srv1655213260:85 , srv1727362576:85 , srv1862016463:85 , srv1950613792:85 , srv2025092740:85 , srv2069423735:85 , srv404579205:85 , srv451866007:85 , srv823760974:85 , srv841229950:85 , srv987748917:85 , srv100266818:86 , srv1047223016:86 , srv1324009065:86 , srv1563032344:86 , srv1707785418:86 , srv2130726093:86 , srv672406718:86 , srv728915235:86 , srv968203370:86 , srv1073625498:87 , srv140007689:87 , srv140238226:87 , srv1490146528:87 , srv150427401:87 , srv1653669488:87 , srv219372408:87 , srv28806147:87 , srv658625345:87 , srv748687804:87 , srv811506009:87 , srv945053995:87 , srv945226867:87 , srv1161486341:88 , srv1166756303:88 , srv1203248938:88 , srv1276792465:88 , srv1812725747:88 , srv1918591489:88 , srv2004344448:88 , srv2085850088:88 , srv2097138400:88 , srv517686089:88 , srv891799089:88 , srv899758779:88 , srv1349073112:89 , srv1762608951:89 , srv1804661470:89 , srv1902376498:89 , srv1935351078:89 , srv2118988742:89 , srv220159090:89 , srv224816288:89 , srv326785800:89 , srv360291700:89 , srv414301021:89 , srv918300185:89 , srv2089021771:90 , srv536650559:90 , srv382499618:91 , srv1534580307:92 , srv1683166178:92 , srv691274278:92 , srv1993535269:99 , srv1036552000:466 }
2023-12-11 12:53:14,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-11 12:53:14,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1653669488=30, srv1862016463=45, srv2118988742=60, srv1837346990=44, srv1166756303=11, srv476293428=75, srv326785800=69, srv1950613792=51, srv859214431=89, srv688111423=81, srv672406718=80, srv1700579752=35, srv1490146528=23, srv2085850088=56, srv728915235=83, srv312173128=68, srv264029095=65, srv219372408=62, srv531285800=77, srv1094764011=6, srv658625345=79, srv295614185=67, srv1683166178=34, srv2088789750=57, srv1237939703=15, srv815450682=86, srv1416993589=21, srv1880368562=46, srv1900763356=47, srv1324009065=17, srv1006523404=1, srv2130726093=61, srv451866007=74, srv1762608951=40, srv517686089=76, srv691274278=82, srv811506009=85, srv918300185=93, srv1655213260=31, srv1200470085=12, srv1161486341=10, srv404579205=72, srv1707785418=36, srv1640211861=29, srv987748917=99, srv945053995=95, srv1201261872=13, srv150427401=24, srv945226867=96, srv913469947=92, srv1073625498=5, srv1678332032=32, srv1423702748=22, srv968203370=97, srv414301021=73, srv1918591489=49, srv940916546=94, srv1778331528=41, srv1727362576=38, srv1276792465=16, srv140007689=19, srv1812725747=43, srv28806147=66, srv983528869=98, srv1563032344=27, srv360291700=70, srv823760974=87, srv2097138400=59, srv1902376498=48, srv100266818=0, srv1727016423=37, srv748687804=84, srv841229950=88, srv1203248938=14, srv140238226=20, srv1534580307=26, srv1993535269=52, srv1154642200=9, srv891799089=90, srv1602780958=28, srv1349073112=18, srv1047223016=4, srv1804661470=42, srv1530204620=25, srv382499618=71, srv1104185981=8, srv220159090=63, srv536650559=78, srv2025092740=54, srv2004344448=53, srv2069423735=55, srv1935351078=50, srv224816288=64, srv1100502586=7, srv2089021771=58, srv1678684432=33, srv1036552000=2, srv1042841076=3, srv899758779=91, srv1759345384=39} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-11 12:53:14,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-11 12:53:14,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-11 12:53:14,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-11 12:53:14,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-11 12:53:14,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-11 12:53:14,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-11 12:53:14,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-11 12:53:14,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-11 12:53:14,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-11 12:53:14,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-11 12:53:14,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-11 12:53:14,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-11 12:53:14,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-11 12:53:14,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-11 12:53:14,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-11 12:53:14,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-11 12:53:14,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-11 12:53:14,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-11 12:53:14,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-11 12:53:14,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-11 12:53:14,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-11 12:53:14,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-11 12:53:14,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-11 12:53:14,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-11 12:53:14,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-11 12:53:14,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-11 12:53:14,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-11 12:53:14,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-11 12:53:14,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-11 12:53:14,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-11 12:53:14,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-11 12:53:14,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-11 12:53:14,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-11 12:53:14,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-11 12:53:14,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-11 12:53:14,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-11 12:53:14,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-11 12:53:14,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-11 12:53:14,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-11 12:53:14,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-11 12:53:14,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-11 12:53:14,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-11 12:53:14,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-11 12:53:14,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-11 12:53:14,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-11 12:53:14,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-11 12:53:14,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-11 12:53:14,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-11 12:53:14,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-11 12:53:14,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-11 12:53:14,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-11 12:53:14,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-11 12:53:14,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-11 12:53:14,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-11 12:53:14,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-11 12:53:14,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-11 12:53:14,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-11 12:53:14,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-11 12:53:14,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-11 12:53:14,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-11 12:53:14,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-11 12:53:14,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-11 12:53:14,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-11 12:53:14,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-11 12:53:14,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-11 12:53:14,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-11 12:53:14,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-11 12:53:14,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-11 12:53:14,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-11 12:53:14,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-11 12:53:14,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-11 12:53:14,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-11 12:53:14,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-11 12:53:14,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-11 12:53:14,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-11 12:53:14,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-11 12:53:14,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-11 12:53:14,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-11 12:53:14,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-11 12:53:14,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-11 12:53:14,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-11 12:53:14,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-11 12:53:14,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-11 12:53:14,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-11 12:53:14,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-11 12:53:14,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-11 12:53:14,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-11 12:53:14,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-11 12:53:14,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-11 12:53:14,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-11 12:53:14,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-11 12:53:14,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-11 12:53:14,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-11 12:53:14,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-11 12:53:14,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-11 12:53:14,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-11 12:53:14,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-11 12:53:14,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-11 12:53:14,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-11 12:53:14,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-11 12:53:14,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-11 12:53:14,909 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04231073479729663, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05728813503042098, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04231073479729663, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17825, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24275, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-11 12:53:14,909 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-12-11 12:53:14,910 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.182790139368472, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04231073479729663, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05728813503042098, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04231073479729663, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17825, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24275, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-12-11 12:53:15,219 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 439 ms to try 5000 different iterations.  Found a solution that moves 2660 regions; Going from a computed imbalance of 0.182790139368472 to a new imbalance of 0.014473240271820535. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023390715610156227); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03612237586537424, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29555555555555557, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023390715610156227); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011416666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043333333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-11 12:53:15,234 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1993535269:72 , srv1349073112:81 , srv1534580307:81 , srv1683166178:81 , srv1902376498:81 , srv220159090:81 , srv517686089:81 , srv691274278:81 , srv140238226:82 , srv1490146528:82 , srv1804661470:82 , srv2118988742:82 , srv326785800:82 , srv414301021:82 , srv536650559:82 , srv100266818:83 , srv1073625498:83 , srv1161486341:83 , srv1762608951:83 , srv2004344448:83 , srv2085850088:83 , srv382499618:83 , srv1047223016:84 , srv1166756303:84 , srv1203248938:84 , srv1276792465:84 , srv150427401:84 , srv1653669488:84 , srv1812725747:84 , srv2089021771:84 , srv2097138400:84 , srv224816288:84 , srv360291700:84 , srv891799089:84 , srv987748917:84 , srv1104185981:85 , srv1918591489:85 , srv1935351078:85 , srv28806147:85 , srv658625345:85 , srv672406718:85 , srv728915235:85 , srv748687804:85 , srv1154642200:86 , srv140007689:86 , srv1655213260:86 , srv1707785418:86 , srv219372408:86 , srv404579205:86 , srv451866007:86 , srv811506009:86 , srv841229950:86 , srv899758779:86 , srv918300185:86 , srv945053995:86 , srv945226867:86 , srv1200470085:87 , srv1324009065:87 , srv1727362576:87 , srv1837346990:87 , srv1862016463:87 , srv531285800:87 , srv859214431:87 , srv968203370:87 , srv1042841076:88 , srv1094764011:88 , srv1201261872:88 , srv1423702748:88 , srv1530204620:88 , srv1563032344:88 , srv1700579752:88 , srv1759345384:88 , srv1778331528:88 , srv1880368562:88 , srv1900763356:88 , srv1950613792:88 , srv2069423735:88 , srv2088789750:88 , srv2130726093:88 , srv476293428:88 , srv815450682:88 , srv823760974:88 , srv983528869:88 , srv1100502586:89 , srv1416993589:89 , srv1678684432:89 , srv1727016423:89 , srv2025092740:89 , srv295614185:89 , srv688111423:89 , srv913469947:89 , srv940916546:89 , srv1006523404:90 , srv1237939703:90 , srv1602780958:90 , srv1640211861:90 , srv1678332032:90 , srv312173128:90 , srv264029095:91 , srv1036552000:516 }
2023-12-11 12:53:15,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-11 12:53:15,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1653669488=30, srv1862016463=45, srv2118988742=60, srv1837346990=44, srv1166756303=11, srv476293428=75, srv326785800=69, srv1950613792=51, srv859214431=89, srv688111423=81, srv672406718=80, srv1700579752=35, srv1490146528=23, srv2085850088=56, srv728915235=83, srv312173128=68, srv264029095=65, srv219372408=62, srv531285800=77, srv1094764011=6, srv658625345=79, srv295614185=67, srv1683166178=34, srv2088789750=57, srv1237939703=15, srv815450682=86, srv1416993589=21, srv1880368562=46, srv1900763356=47, srv1324009065=17, srv1006523404=1, srv2130726093=61, srv451866007=74, srv1762608951=40, srv517686089=76, srv691274278=82, srv811506009=85, srv918300185=93, srv1655213260=31, srv1200470085=12, srv1161486341=10, srv404579205=72, srv1707785418=36, srv1640211861=29, srv987748917=99, srv945053995=95, srv1201261872=13, srv150427401=24, srv945226867=96, srv913469947=92, srv1073625498=5, srv1678332032=32, srv1423702748=22, srv968203370=97, srv414301021=73, srv1918591489=49, srv940916546=94, srv1778331528=41, srv1727362576=38, srv1276792465=16, srv140007689=19, srv1812725747=43, srv28806147=66, srv983528869=98, srv1563032344=27, srv360291700=70, srv823760974=87, srv2097138400=59, srv1902376498=48, srv100266818=0, srv1727016423=37, srv748687804=84, srv841229950=88, srv1203248938=14, srv140238226=20, srv1534580307=26, srv1993535269=52, srv1154642200=9, srv891799089=90, srv1602780958=28, srv1349073112=18, srv1047223016=4, srv1804661470=42, srv1530204620=25, srv382499618=71, srv1104185981=8, srv220159090=63, srv536650559=78, srv2025092740=54, srv2004344448=53, srv2069423735=55, srv1935351078=50, srv224816288=64, srv1100502586=7, srv2089021771=58, srv1678684432=33, srv1036552000=2, srv1042841076=3, srv899758779=91, srv1759345384=39} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-11 12:53:15,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-11 12:53:15,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-11 12:53:15,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-11 12:53:15,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-11 12:53:15,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-11 12:53:15,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-11 12:53:15,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-11 12:53:15,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-11 12:53:15,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-11 12:53:15,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-11 12:53:15,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-11 12:53:15,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-11 12:53:15,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-11 12:53:15,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-11 12:53:15,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-11 12:53:15,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-11 12:53:15,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-11 12:53:15,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-11 12:53:15,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-11 12:53:15,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-11 12:53:15,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-11 12:53:15,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-11 12:53:15,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-11 12:53:15,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-11 12:53:15,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-11 12:53:15,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-11 12:53:15,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-11 12:53:15,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-11 12:53:15,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-11 12:53:15,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-11 12:53:15,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-11 12:53:15,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-11 12:53:15,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-11 12:53:15,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-11 12:53:15,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-11 12:53:15,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-11 12:53:15,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-11 12:53:15,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-11 12:53:15,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-11 12:53:15,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-11 12:53:15,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-11 12:53:15,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-11 12:53:15,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-11 12:53:15,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-11 12:53:15,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-11 12:53:15,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-11 12:53:15,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-11 12:53:15,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-11 12:53:15,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-11 12:53:15,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-11 12:53:15,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-11 12:53:15,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-11 12:53:15,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-11 12:53:15,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-11 12:53:15,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-11 12:53:15,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-11 12:53:15,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-11 12:53:15,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-11 12:53:15,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-11 12:53:15,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-11 12:53:15,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-11 12:53:15,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-11 12:53:15,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-11 12:53:15,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-11 12:53:15,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-11 12:53:15,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-11 12:53:15,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-11 12:53:15,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-11 12:53:15,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-11 12:53:15,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-11 12:53:15,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-11 12:53:15,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-11 12:53:15,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-11 12:53:15,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-11 12:53:15,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-11 12:53:15,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-11 12:53:15,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-11 12:53:15,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-11 12:53:15,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-11 12:53:15,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-11 12:53:15,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-11 12:53:15,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-11 12:53:15,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-11 12:53:15,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-11 12:53:15,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-11 12:53:15,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-11 12:53:15,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-11 12:53:15,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-11 12:53:15,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-11 12:53:15,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-11 12:53:15,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-11 12:53:15,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-11 12:53:15,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-11 12:53:15,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-11 12:53:15,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-11 12:53:15,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-11 12:53:15,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-11 12:53:15,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-11 12:53:15,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-11 12:53:15,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-11 12:53:15,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-11 12:53:15,437 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023390715610156227); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03612237586537424, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023390715610156227); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011416666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043333333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-11 12:53:15,437 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-12-11 12:53:15,438 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.014454614553725082, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023390715610156227); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03612237586537424, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023390715610156227); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011416666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043333333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-12-11 12:53:15,842 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 547 ms to try 5000 different iterations.  Found a solution that moves 976 regions; Going from a computed imbalance of 0.014454614553725082 to a new imbalance of 9.947943669270727E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006776130873858127); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.010562198866399846); MoveCostFunction : (multiplier=7.0, imbalance=0.10844444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006776130873858127); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.0E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.005083333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-11 12:53:15,844 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1602780958:83 , srv1678684432:83 , srv2130726093:83 , srv918300185:83 , srv945053995:83 , srv1006523404:84 , srv1201261872:84 , srv1324009065:84 , srv140007689:84 , srv1416993589:84 , srv1423702748:84 , srv1534580307:84 , srv1563032344:84 , srv1683166178:84 , srv1700579752:84 , srv2025092740:84 , srv2088789750:84 , srv2089021771:84 , srv219372408:84 , srv295614185:84 , srv312173128:84 , srv382499618:84 , srv688111423:84 , srv823760974:84 , srv899758779:84 , srv940916546:84 , srv945226867:84 , srv1042841076:85 , srv1200470085:85 , srv1203248938:85 , srv1276792465:85 , srv150427401:85 , srv1530204620:85 , srv1678332032:85 , srv1707785418:85 , srv1727016423:85 , srv1727362576:85 , srv1862016463:85 , srv1880368562:85 , srv1918591489:85 , srv1935351078:85 , srv1950613792:85 , srv2069423735:85 , srv224816288:85 , srv264029095:85 , srv28806147:85 , srv360291700:85 , srv658625345:85 , srv691274278:85 , srv811506009:85 , srv815450682:85 , srv891799089:85 , srv913469947:85 , srv968203370:85 , srv1094764011:86 , srv1100502586:86 , srv1154642200:86 , srv1166756303:86 , srv1237939703:86 , srv1640211861:86 , srv1655213260:86 , srv1759345384:86 , srv1762608951:86 , srv1804661470:86 , srv1812725747:86 , srv1900763356:86 , srv1993535269:86 , srv2004344448:86 , srv2085850088:86 , srv2097138400:86 , srv2118988742:86 , srv414301021:86 , srv451866007:86 , srv476293428:86 , srv536650559:86 , srv672406718:86 , srv728915235:86 , srv748687804:86 , srv841229950:86 , srv983528869:86 , srv1073625498:87 , srv1161486341:87 , srv1653669488:87 , srv1902376498:87 , srv326785800:87 , srv404579205:87 , srv859214431:87 , srv100266818:88 , srv1047223016:88 , srv1778331528:88 , srv1837346990:88 , srv220159090:88 , srv531285800:88 , srv987748917:88 , srv1104185981:89 , srv1349073112:89 , srv140238226:89 , srv1490146528:89 , srv517686089:89 , srv1036552000:536 }
2023-12-11 12:53:15,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-11 12:53:15,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1653669488=30, srv1862016463=45, srv2118988742=60, srv1837346990=44, srv1166756303=11, srv476293428=75, srv326785800=69, srv1950613792=51, srv859214431=89, srv688111423=81, srv672406718=80, srv1700579752=35, srv1490146528=23, srv2085850088=56, srv728915235=83, srv312173128=68, srv264029095=65, srv219372408=62, srv531285800=77, srv1094764011=6, srv658625345=79, srv295614185=67, srv1683166178=34, srv2088789750=57, srv1237939703=15, srv815450682=86, srv1416993589=21, srv1880368562=46, srv1900763356=47, srv1324009065=17, srv1006523404=1, srv2130726093=61, srv451866007=74, srv1762608951=40, srv517686089=76, srv691274278=82, srv811506009=85, srv918300185=93, srv1655213260=31, srv1200470085=12, srv1161486341=10, srv404579205=72, srv1707785418=36, srv1640211861=29, srv987748917=99, srv945053995=95, srv1201261872=13, srv150427401=24, srv945226867=96, srv913469947=92, srv1073625498=5, srv1678332032=32, srv1423702748=22, srv968203370=97, srv414301021=73, srv1918591489=49, srv940916546=94, srv1778331528=41, srv1727362576=38, srv1276792465=16, srv140007689=19, srv1812725747=43, srv28806147=66, srv983528869=98, srv1563032344=27, srv360291700=70, srv823760974=87, srv2097138400=59, srv1902376498=48, srv100266818=0, srv1727016423=37, srv748687804=84, srv841229950=88, srv1203248938=14, srv140238226=20, srv1534580307=26, srv1993535269=52, srv1154642200=9, srv891799089=90, srv1602780958=28, srv1349073112=18, srv1047223016=4, srv1804661470=42, srv1530204620=25, srv382499618=71, srv1104185981=8, srv220159090=63, srv536650559=78, srv2025092740=54, srv2004344448=53, srv2069423735=55, srv1935351078=50, srv224816288=64, srv1100502586=7, srv2089021771=58, srv1678684432=33, srv1036552000=2, srv1042841076=3, srv899758779=91, srv1759345384=39} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-11 12:53:15,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-11 12:53:15,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-11 12:53:15,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-11 12:53:15,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-11 12:53:15,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-11 12:53:15,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-11 12:53:15,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-11 12:53:15,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-11 12:53:15,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-11 12:53:15,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-11 12:53:15,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-11 12:53:15,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-11 12:53:15,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-11 12:53:15,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-11 12:53:15,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-11 12:53:15,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-11 12:53:15,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-11 12:53:15,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-11 12:53:15,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-11 12:53:15,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-11 12:53:15,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-11 12:53:15,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-11 12:53:15,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-11 12:53:15,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-11 12:53:15,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-11 12:53:15,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-11 12:53:15,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-11 12:53:15,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-11 12:53:15,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-11 12:53:15,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-11 12:53:15,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-11 12:53:15,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-11 12:53:15,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-11 12:53:15,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-11 12:53:15,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-11 12:53:15,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-11 12:53:15,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-11 12:53:15,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-11 12:53:15,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-11 12:53:15,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-11 12:53:15,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-11 12:53:15,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-11 12:53:15,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-11 12:53:15,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-11 12:53:15,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-11 12:53:15,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-11 12:53:15,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-11 12:53:15,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-11 12:53:15,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-11 12:53:15,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-11 12:53:15,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-11 12:53:15,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-11 12:53:15,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-11 12:53:15,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-11 12:53:15,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-11 12:53:15,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-11 12:53:15,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-11 12:53:15,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-11 12:53:15,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-11 12:53:15,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-11 12:53:15,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-11 12:53:15,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-11 12:53:15,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-11 12:53:15,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-11 12:53:15,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-11 12:53:15,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-11 12:53:15,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-11 12:53:15,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-11 12:53:15,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-11 12:53:15,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-11 12:53:15,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-11 12:53:15,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-11 12:53:15,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-11 12:53:15,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-11 12:53:15,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-11 12:53:15,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-11 12:53:15,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-11 12:53:15,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-11 12:53:15,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-11 12:53:15,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-11 12:53:15,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-11 12:53:15,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-11 12:53:15,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-11 12:53:15,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-11 12:53:15,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-11 12:53:15,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-11 12:53:15,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-11 12:53:15,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-11 12:53:15,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-11 12:53:15,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-11 12:53:15,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-11 12:53:15,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-11 12:53:15,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-11 12:53:15,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-11 12:53:15,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-11 12:53:15,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-11 12:53:15,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-11 12:53:15,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-11 12:53:15,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-11 12:53:15,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-11 12:53:15,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-11 12:53:15,972 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006776130873858127); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.010562198866399846); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006776130873858127); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.0E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.005083333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-11 12:53:15,972 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-12-11 12:53:15,972 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=9.879602688589659E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006776130873858127); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.010562198866399846); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006776130873858127); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.0E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.005083333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-12-11 12:53:16,251 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 377 ms to try 5000 different iterations.  Found a solution that moves 290 regions; Going from a computed imbalance of 9.879602688589659E-4 to a new imbalance of 4.410017883358985E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.23782800878924E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002118805753879094); MoveCostFunction : (multiplier=7.0, imbalance=0.03222222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.23782800878924E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-11 12:53:16,253 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1104185981:83 , srv1349073112:83 , srv1653669488:83 , srv1935351078:83 , srv360291700:83 , srv1047223016:84 , srv1094764011:84 , srv1100502586:84 , srv1161486341:84 , srv1166756303:84 , srv1203248938:84 , srv1237939703:84 , srv140238226:84 , srv1490146528:84 , srv1534580307:84 , srv1563032344:84 , srv1678332032:84 , srv1727016423:84 , srv1727362576:84 , srv1759345384:84 , srv1762608951:84 , srv1804661470:84 , srv1837346990:84 , srv1902376498:84 , srv1950613792:84 , srv2025092740:84 , srv2069423735:84 , srv2089021771:84 , srv220159090:84 , srv264029095:84 , srv28806147:84 , srv326785800:84 , srv404579205:84 , srv414301021:84 , srv476293428:84 , srv517686089:84 , srv531285800:84 , srv536650559:84 , srv691274278:84 , srv748687804:84 , srv811506009:84 , srv815450682:84 , srv891799089:84 , srv899758779:84 , srv918300185:84 , srv968203370:84 , srv983528869:84 , srv100266818:85 , srv1006523404:85 , srv1042841076:85 , srv1073625498:85 , srv1154642200:85 , srv1201261872:85 , srv1276792465:85 , srv1324009065:85 , srv140007689:85 , srv1423702748:85 , srv1530204620:85 , srv1602780958:85 , srv1640211861:85 , srv1655213260:85 , srv1683166178:85 , srv1707785418:85 , srv1778331528:85 , srv1812725747:85 , srv1880368562:85 , srv1900763356:85 , srv1918591489:85 , srv1993535269:85 , srv2004344448:85 , srv2085850088:85 , srv2097138400:85 , srv2118988742:85 , srv219372408:85 , srv224816288:85 , srv312173128:85 , srv382499618:85 , srv451866007:85 , srv672406718:85 , srv688111423:85 , srv823760974:85 , srv841229950:85 , srv859214431:85 , srv940916546:85 , srv945226867:85 , srv987748917:85 , srv1200470085:86 , srv1416993589:86 , srv150427401:86 , srv1678684432:86 , srv1700579752:86 , srv1862016463:86 , srv2088789750:86 , srv2130726093:86 , srv295614185:86 , srv658625345:86 , srv728915235:86 , srv913469947:86 , srv945053995:86 , srv1036552000:624 }
2023-12-11 12:53:16,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-11 12:53:16,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1653669488=30, srv1862016463=45, srv2118988742=60, srv1837346990=44, srv1166756303=11, srv476293428=75, srv326785800=69, srv1950613792=51, srv859214431=89, srv688111423=81, srv672406718=80, srv1700579752=35, srv1490146528=23, srv2085850088=56, srv728915235=83, srv312173128=68, srv264029095=65, srv219372408=62, srv531285800=77, srv1094764011=6, srv658625345=79, srv295614185=67, srv1683166178=34, srv2088789750=57, srv1237939703=15, srv815450682=86, srv1416993589=21, srv1880368562=46, srv1900763356=47, srv1324009065=17, srv1006523404=1, srv2130726093=61, srv451866007=74, srv1762608951=40, srv517686089=76, srv691274278=82, srv811506009=85, srv918300185=93, srv1655213260=31, srv1200470085=12, srv1161486341=10, srv404579205=72, srv1707785418=36, srv1640211861=29, srv987748917=99, srv945053995=95, srv1201261872=13, srv150427401=24, srv945226867=96, srv913469947=92, srv1073625498=5, srv1678332032=32, srv1423702748=22, srv968203370=97, srv414301021=73, srv1918591489=49, srv940916546=94, srv1778331528=41, srv1727362576=38, srv1276792465=16, srv140007689=19, srv1812725747=43, srv28806147=66, srv983528869=98, srv1563032344=27, srv360291700=70, srv823760974=87, srv2097138400=59, srv1902376498=48, srv100266818=0, srv1727016423=37, srv748687804=84, srv841229950=88, srv1203248938=14, srv140238226=20, srv1534580307=26, srv1993535269=52, srv1154642200=9, srv891799089=90, srv1602780958=28, srv1349073112=18, srv1047223016=4, srv1804661470=42, srv1530204620=25, srv382499618=71, srv1104185981=8, srv220159090=63, srv536650559=78, srv2025092740=54, srv2004344448=53, srv2069423735=55, srv1935351078=50, srv224816288=64, srv1100502586=7, srv2089021771=58, srv1678684432=33, srv1036552000=2, srv1042841076=3, srv899758779=91, srv1759345384=39} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-11 12:53:16,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-11 12:53:16,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-11 12:53:16,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-11 12:53:16,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-11 12:53:16,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-11 12:53:16,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-11 12:53:16,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-11 12:53:16,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-11 12:53:16,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-11 12:53:16,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-11 12:53:16,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-11 12:53:16,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-11 12:53:16,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-11 12:53:16,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-11 12:53:16,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-11 12:53:16,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-11 12:53:16,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-11 12:53:16,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-11 12:53:16,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-11 12:53:16,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-11 12:53:16,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-11 12:53:16,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-11 12:53:16,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-11 12:53:16,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-11 12:53:16,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-11 12:53:16,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-11 12:53:16,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-11 12:53:16,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-11 12:53:16,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-11 12:53:16,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-11 12:53:16,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-11 12:53:16,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-11 12:53:16,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-11 12:53:16,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-11 12:53:16,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-11 12:53:16,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-11 12:53:16,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-11 12:53:16,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-11 12:53:16,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-11 12:53:16,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-11 12:53:16,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-11 12:53:16,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-11 12:53:16,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-11 12:53:16,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-11 12:53:16,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-11 12:53:16,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-11 12:53:16,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-11 12:53:16,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-11 12:53:16,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-11 12:53:16,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-11 12:53:16,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-11 12:53:16,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-11 12:53:16,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-11 12:53:16,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-11 12:53:16,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-11 12:53:16,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-11 12:53:16,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-11 12:53:16,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-11 12:53:16,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-11 12:53:16,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-11 12:53:16,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-11 12:53:16,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-11 12:53:16,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-11 12:53:16,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-11 12:53:16,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-11 12:53:16,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-11 12:53:16,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-11 12:53:16,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-11 12:53:16,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-11 12:53:16,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-11 12:53:16,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-11 12:53:16,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-11 12:53:16,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-11 12:53:16,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-11 12:53:16,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-11 12:53:16,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-11 12:53:16,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-11 12:53:16,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-11 12:53:16,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-11 12:53:16,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-11 12:53:16,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-11 12:53:16,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-11 12:53:16,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-11 12:53:16,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-11 12:53:16,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-11 12:53:16,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-11 12:53:16,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-11 12:53:16,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-11 12:53:16,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-11 12:53:16,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-11 12:53:16,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-11 12:53:16,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-11 12:53:16,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-11 12:53:16,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-11 12:53:16,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-11 12:53:16,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-11 12:53:16,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-11 12:53:16,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-11 12:53:16,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-11 12:53:16,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-11 12:53:16,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-11 12:53:16,409 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.23782800878924E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002118805753879094); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.23782800878924E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-11 12:53:16,409 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-12-11 12:53:16,410 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=4.206955543220563E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.23782800878924E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002118805753879094); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.23782800878924E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-12-11 12:53:16,677 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 385 ms to try 5000 different iterations.  Found a solution that moves 36 regions; Going from a computed imbalance of 4.206955543220563E-5 to a new imbalance of 7.639834888070601E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0016412198797244363); MoveCostFunction : (multiplier=7.0, imbalance=0.004); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-11 12:53:16,678 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1778331528:83 , srv1812725747:83 , srv1862016463:83 , srv1918591489:83 , srv2097138400:83 , srv2130726093:83 , srv224816288:83 , srv295614185:83 , srv382499618:83 , srv658625345:83 , srv728915235:83 , srv100266818:84 , srv1006523404:84 , srv1042841076:84 , srv1047223016:84 , srv1073625498:84 , srv1094764011:84 , srv1100502586:84 , srv1104185981:84 , srv1154642200:84 , srv1161486341:84 , srv1166756303:84 , srv1200470085:84 , srv1201261872:84 , srv1237939703:84 , srv1276792465:84 , srv1324009065:84 , srv1349073112:84 , srv140007689:84 , srv140238226:84 , srv1416993589:84 , srv1423702748:84 , srv1490146528:84 , srv150427401:84 , srv1530204620:84 , srv1563032344:84 , srv1602780958:84 , srv1640211861:84 , srv1655213260:84 , srv1678332032:84 , srv1678684432:84 , srv1683166178:84 , srv1700579752:84 , srv1707785418:84 , srv1727016423:84 , srv1759345384:84 , srv1762608951:84 , srv1837346990:84 , srv1880368562:84 , srv1900763356:84 , srv1935351078:84 , srv1950613792:84 , srv1993535269:84 , srv2004344448:84 , srv2025092740:84 , srv2069423735:84 , srv2085850088:84 , srv2088789750:84 , srv2089021771:84 , srv2118988742:84 , srv219372408:84 , srv220159090:84 , srv264029095:84 , srv312173128:84 , srv326785800:84 , srv404579205:84 , srv451866007:84 , srv476293428:84 , srv517686089:84 , srv531285800:84 , srv536650559:84 , srv672406718:84 , srv688111423:84 , srv691274278:84 , srv748687804:84 , srv811506009:84 , srv823760974:84 , srv841229950:84 , srv859214431:84 , srv899758779:84 , srv913469947:84 , srv918300185:84 , srv940916546:84 , srv945053995:84 , srv945226867:84 , srv968203370:84 , srv983528869:84 , srv987748917:84 , srv1203248938:85 , srv1534580307:85 , srv1653669488:85 , srv1727362576:85 , srv1804661470:85 , srv1902376498:85 , srv28806147:85 , srv360291700:85 , srv414301021:85 , srv815450682:85 , srv891799089:85 , srv1036552000:684 }
2023-12-11 12:53:16,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-11 12:53:16,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1653669488=30, srv1862016463=45, srv2118988742=60, srv1837346990=44, srv1166756303=11, srv476293428=75, srv326785800=69, srv1950613792=51, srv859214431=89, srv688111423=81, srv672406718=80, srv1700579752=35, srv1490146528=23, srv2085850088=56, srv728915235=83, srv312173128=68, srv264029095=65, srv219372408=62, srv531285800=77, srv1094764011=6, srv658625345=79, srv295614185=67, srv1683166178=34, srv2088789750=57, srv1237939703=15, srv815450682=86, srv1416993589=21, srv1880368562=46, srv1900763356=47, srv1324009065=17, srv1006523404=1, srv2130726093=61, srv451866007=74, srv1762608951=40, srv517686089=76, srv691274278=82, srv811506009=85, srv918300185=93, srv1655213260=31, srv1200470085=12, srv1161486341=10, srv404579205=72, srv1707785418=36, srv1640211861=29, srv987748917=99, srv945053995=95, srv1201261872=13, srv150427401=24, srv945226867=96, srv913469947=92, srv1073625498=5, srv1678332032=32, srv1423702748=22, srv968203370=97, srv414301021=73, srv1918591489=49, srv940916546=94, srv1778331528=41, srv1727362576=38, srv1276792465=16, srv140007689=19, srv1812725747=43, srv28806147=66, srv983528869=98, srv1563032344=27, srv360291700=70, srv823760974=87, srv2097138400=59, srv1902376498=48, srv100266818=0, srv1727016423=37, srv748687804=84, srv841229950=88, srv1203248938=14, srv140238226=20, srv1534580307=26, srv1993535269=52, srv1154642200=9, srv891799089=90, srv1602780958=28, srv1349073112=18, srv1047223016=4, srv1804661470=42, srv1530204620=25, srv382499618=71, srv1104185981=8, srv220159090=63, srv536650559=78, srv2025092740=54, srv2004344448=53, srv2069423735=55, srv1935351078=50, srv224816288=64, srv1100502586=7, srv2089021771=58, srv1678684432=33, srv1036552000=2, srv1042841076=3, srv899758779=91, srv1759345384=39} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-11 12:53:16,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-11 12:53:16,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-11 12:53:16,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-11 12:53:16,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-11 12:53:16,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-11 12:53:16,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-11 12:53:16,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-11 12:53:16,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-11 12:53:16,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-11 12:53:16,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-11 12:53:16,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-11 12:53:16,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-11 12:53:16,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-11 12:53:16,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-11 12:53:16,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-11 12:53:16,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-11 12:53:16,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-11 12:53:16,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-11 12:53:16,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-11 12:53:16,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-11 12:53:16,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-11 12:53:16,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-11 12:53:16,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-11 12:53:16,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-11 12:53:16,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-11 12:53:16,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-11 12:53:16,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-11 12:53:16,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-11 12:53:16,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-11 12:53:16,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-11 12:53:16,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-11 12:53:16,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-11 12:53:16,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-11 12:53:16,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-11 12:53:16,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-11 12:53:16,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-11 12:53:16,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-11 12:53:16,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-11 12:53:16,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-11 12:53:16,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-11 12:53:16,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-11 12:53:16,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-11 12:53:16,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-11 12:53:16,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-11 12:53:16,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-11 12:53:16,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-11 12:53:16,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-11 12:53:16,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-11 12:53:16,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-11 12:53:16,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-11 12:53:16,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-11 12:53:16,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-11 12:53:16,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-11 12:53:16,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-11 12:53:16,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-11 12:53:16,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-11 12:53:16,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-11 12:53:16,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-11 12:53:16,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-11 12:53:16,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-11 12:53:16,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-11 12:53:16,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-11 12:53:16,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-11 12:53:16,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-11 12:53:16,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-11 12:53:16,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-11 12:53:16,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-11 12:53:16,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-11 12:53:16,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-11 12:53:16,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-11 12:53:16,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-11 12:53:16,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-11 12:53:16,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-11 12:53:16,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-11 12:53:16,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-11 12:53:16,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-11 12:53:16,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-11 12:53:16,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-11 12:53:16,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-11 12:53:16,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-11 12:53:16,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-11 12:53:16,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-11 12:53:16,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-11 12:53:16,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-11 12:53:16,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-11 12:53:16,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-11 12:53:16,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-11 12:53:16,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-11 12:53:16,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-11 12:53:16,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-11 12:53:16,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-11 12:53:16,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-11 12:53:16,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-11 12:53:16,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-11 12:53:16,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-11 12:53:16,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-11 12:53:16,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-11 12:53:16,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-11 12:53:16,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-11 12:53:16,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-11 12:53:16,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-11 12:53:16,856 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=7.387757500312559E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0016412198797244363); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-11 12:53:16,857 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1778331528:83 , srv1812725747:83 , srv1862016463:83 , srv1918591489:83 , srv2097138400:83 , srv2130726093:83 , srv224816288:83 , srv295614185:83 , srv382499618:83 , srv658625345:83 , srv728915235:83 , srv100266818:84 , srv1006523404:84 , srv1042841076:84 , srv1047223016:84 , srv1073625498:84 , srv1094764011:84 , srv1100502586:84 , srv1104185981:84 , srv1154642200:84 , srv1161486341:84 , srv1166756303:84 , srv1200470085:84 , srv1201261872:84 , srv1237939703:84 , srv1276792465:84 , srv1324009065:84 , srv1349073112:84 , srv140007689:84 , srv140238226:84 , srv1416993589:84 , srv1423702748:84 , srv1490146528:84 , srv150427401:84 , srv1530204620:84 , srv1563032344:84 , srv1602780958:84 , srv1640211861:84 , srv1655213260:84 , srv1678332032:84 , srv1678684432:84 , srv1683166178:84 , srv1700579752:84 , srv1707785418:84 , srv1727016423:84 , srv1759345384:84 , srv1762608951:84 , srv1837346990:84 , srv1880368562:84 , srv1900763356:84 , srv1935351078:84 , srv1950613792:84 , srv1993535269:84 , srv2004344448:84 , srv2025092740:84 , srv2069423735:84 , srv2085850088:84 , srv2088789750:84 , srv2089021771:84 , srv2118988742:84 , srv219372408:84 , srv220159090:84 , srv264029095:84 , srv312173128:84 , srv326785800:84 , srv404579205:84 , srv451866007:84 , srv476293428:84 , srv517686089:84 , srv531285800:84 , srv536650559:84 , srv672406718:84 , srv688111423:84 , srv691274278:84 , srv748687804:84 , srv811506009:84 , srv823760974:84 , srv841229950:84 , srv859214431:84 , srv899758779:84 , srv913469947:84 , srv918300185:84 , srv940916546:84 , srv945053995:84 , srv945226867:84 , srv968203370:84 , srv983528869:84 , srv987748917:84 , srv1203248938:85 , srv1534580307:85 , srv1653669488:85 , srv1727362576:85 , srv1804661470:85 , srv1902376498:85 , srv28806147:85 , srv360291700:85 , srv414301021:85 , srv815450682:85 , srv891799089:85 , srv1036552000:684 }
2023-12-11 12:53:16,903 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=281 (was 281), ProcessCount=171 (was 171), AvailableMemoryMB=5720 (was 6651)
2023-12-11 12:53:16,914 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=281, ProcessCount=171, AvailableMemoryMB=5719
2023-12-11 12:53:16,915 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-12-11 12:53:16,915 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-12-11 12:53:16,933 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1128570714:84 , srv1149448423:84 , srv1150549331:84 , srv1279853156:84 , srv1356119314:84 , srv1513949745:84 , srv1587696857:84 , srv1638627164:84 , srv1668343880:84 , srv1695568171:84 , srv173639204:84 , srv1863276182:84 , srv1925417706:84 , srv1951444843:84 , srv1983500426:84 , srv2056135207:84 , srv265609344:84 , srv325173039:84 , srv414938334:84 , srv41580547:84 , srv418075358:84 , srv430402455:84 , srv497729284:84 , srv56016107:84 , srv570056201:84 , srv695954831:84 , srv728270951:84 , srv954526349:84 , srv963243492:84 , srv255387466:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-12-11 12:53:16,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-11 12:53:16,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv56016107=24, srv1983500426=14, srv265609344=17, srv414938334=19, srv954526349=28, srv325173039=18, srv1638627164=7, srv1128570714=0, srv570056201=25, srv1695568171=9, srv2056135207=15, srv497729284=23, srv1925417706=12, srv1149448423=1, srv1668343880=8, srv255387466=16, srv418075358=21, srv728270951=27, srv1863276182=11, srv1951444843=13, srv1150549331=2, srv1279853156=3, srv173639204=10, srv1587696857=6, srv1513949745=5, srv41580547=20, srv695954831=26, srv1356119314=4, srv963243492=29, srv430402455=22} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-11 12:53:16,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-11 12:53:16,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-11 12:53:16,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-11 12:53:16,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-11 12:53:16,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-11 12:53:16,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-11 12:53:16,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-11 12:53:16,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-11 12:53:16,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-11 12:53:16,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-11 12:53:16,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-11 12:53:16,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-11 12:53:16,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-11 12:53:16,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-11 12:53:16,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-11 12:53:16,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-11 12:53:16,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-11 12:53:16,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-11 12:53:16,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-11 12:53:16,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-11 12:53:16,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-11 12:53:16,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-11 12:53:16,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-11 12:53:16,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-11 12:53:16,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-11 12:53:16,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-11 12:53:16,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-11 12:53:16,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-11 12:53:16,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-11 12:53:16,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-11 12:53:16,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-12-11 12:53:16,971 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1452874972160192, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-11 12:53:16,971 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912650839423934, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1452874972160192, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-12-11 12:55:16,955 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5694117 different iterations.  Found a solution that moves 1858 regions; Going from a computed imbalance of 0.9912650839423934 to a new imbalance of 4.336664689392977E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6881481481481482, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-11 12:55:16,960 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1128570714:90 , srv1149448423:90 , srv1150549331:90 , srv1279853156:90 , srv1356119314:90 , srv1513949745:90 , srv1587696857:90 , srv1638627164:90 , srv1668343880:90 , srv1695568171:90 , srv173639204:90 , srv1863276182:90 , srv1925417706:90 , srv1951444843:90 , srv1983500426:90 , srv2056135207:90 , srv255387466:90 , srv265609344:90 , srv325173039:90 , srv414938334:90 , srv41580547:90 , srv418075358:90 , srv430402455:90 , srv497729284:90 , srv56016107:90 , srv570056201:90 , srv695954831:90 , srv728270951:90 , srv954526349:90 , srv963243492:90 }
2023-12-11 12:55:16,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-11 12:55:16,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv56016107=24, srv1983500426=14, srv265609344=17, srv414938334=19, srv954526349=28, srv325173039=18, srv1638627164=7, srv1128570714=0, srv570056201=25, srv1695568171=9, srv2056135207=15, srv497729284=23, srv1925417706=12, srv1149448423=1, srv1668343880=8, srv255387466=16, srv418075358=21, srv728270951=27, srv1863276182=11, srv1951444843=13, srv1150549331=2, srv1279853156=3, srv173639204=10, srv1587696857=6, srv1513949745=5, srv41580547=20, srv695954831=26, srv1356119314=4, srv963243492=29, srv430402455=22} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-11 12:55:16,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-11 12:55:16,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-11 12:55:16,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-11 12:55:16,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-11 12:55:16,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-11 12:55:16,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-11 12:55:16,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-11 12:55:16,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-11 12:55:16,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-11 12:55:16,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-11 12:55:16,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-11 12:55:16,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-11 12:55:16,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-11 12:55:16,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-11 12:55:16,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-11 12:55:16,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-11 12:55:16,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-11 12:55:16,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-11 12:55:16,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-11 12:55:16,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-11 12:55:16,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-11 12:55:16,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-11 12:55:16,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-11 12:55:16,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-11 12:55:16,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-11 12:55:16,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-11 12:55:16,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-11 12:55:16,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-11 12:55:16,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-11 12:55:16,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-11 12:55:16,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-12-11 12:55:16,978 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-11 12:55:16,978 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1128570714:90 , srv1149448423:90 , srv1150549331:90 , srv1279853156:90 , srv1356119314:90 , srv1513949745:90 , srv1587696857:90 , srv1638627164:90 , srv1668343880:90 , srv1695568171:90 , srv173639204:90 , srv1863276182:90 , srv1925417706:90 , srv1951444843:90 , srv1983500426:90 , srv2056135207:90 , srv255387466:90 , srv265609344:90 , srv325173039:90 , srv414938334:90 , srv41580547:90 , srv418075358:90 , srv430402455:90 , srv497729284:90 , srv56016107:90 , srv570056201:90 , srv695954831:90 , srv728270951:90 , srv954526349:90 , srv963243492:90 }
2023-12-11 12:55:16,991 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=215 (was 281), ProcessCount=168 (was 171), AvailableMemoryMB=4921 (was 5719)