2023-10-15 19:53:27,893 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-10-15 19:53:28,242 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-10-15 19:53:28,380 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-10-15 19:53:28,448 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-10-15 19:53:28,456 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-10-15 19:53:28,470 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=323, ProcessCount=170, AvailableMemoryMB=6648
2023-10-15 19:53:28,472 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-10-15 19:53:28,472 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-10-15 19:53:28,614 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1007986791:84 , srv1041090471:84 , srv1044735449:84 , srv1052887025:84 , srv1055374215:84 , srv1071801527:84 , srv1091695394:84 , srv1095082718:84 , srv1150743170:84 , srv1179842772:84 , srv1185455147:84 , srv1224647658:84 , srv1236984426:84 , srv1253019227:84 , srv1261063185:84 , srv1281205085:84 , srv1320749546:84 , srv1430771729:84 , srv1441289698:84 , srv1455230607:84 , srv1474899482:84 , srv1514574662:84 , srv153034324:84 , srv1588550243:84 , srv1595009264:84 , srv1609279463:84 , srv161579350:84 , srv1615999575:84 , srv1648253456:84 , srv1648686423:84 , srv1652935864:84 , srv1693027323:84 , srv1726876634:84 , srv1727294131:84 , srv1731747001:84 , srv1732510198:84 , srv1739223862:84 , srv1744666379:84 , srv1751166597:84 , srv1771335643:84 , srv1784586942:84 , srv1787581600:84 , srv1806580104:84 , srv1821175562:84 , srv1825060679:84 , srv1840211266:84 , srv1853867956:84 , srv1873131725:84 , srv1877782356:84 , srv1892224385:84 , srv189909834:84 , srv1908447515:84 , srv1924069238:84 , srv1972966800:84 , srv1996549164:84 , srv202385378:84 , srv2036368900:84 , srv2051788734:84 , srv2064493420:84 , srv2120382996:84 , srv2122796101:84 , srv2124385648:84 , srv2131095941:84 , srv237696716:84 , srv268114086:84 , srv299178434:84 , srv306083069:84 , srv319629628:84 , srv322545395:84 , srv325432250:84 , srv417959332:84 , srv444799157:84 , srv501177272:84 , srv553105152:84 , srv562953736:84 , srv575526193:84 , srv582194224:84 , srv609447077:84 , srv675508027:84 , srv694235394:84 , srv709129020:84 , srv738531535:84 , srv742743553:84 , srv749023457:84 , srv772329470:84 , srv776123721:84 , srv776638132:84 , srv787756481:84 , srv873551319:84 , srv881451098:84 , srv90110108:84 , srv933045416:84 , srv939138896:84 , srv945210971:84 , srv947904493:84 , srv949432644:84 , srv967729145:84 , srv97714977:84 , srv998431403:84 , srv632053504:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-10-15 19:53:28,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-15 19:53:28,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1806580104=42, srv1474899482=20, srv632053504=78, srv1877782356=48, srv90110108=91, srv1739223862=36, srv1924069238=52, srv1071801527=5, srv1253019227=13, srv1320749546=16, srv2131095941=62, srv1892224385=49, srv1595009264=24, srv1726876634=32, srv1744666379=37, srv1179842772=9, srv776638132=87, srv939138896=93, srv2036368900=56, srv1648253456=28, srv738531535=82, srv1091695394=6, srv1007986791=0, srv1224647658=11, srv1784586942=40, srv1751166597=38, srv787756481=88, srv2122796101=60, srv1041090471=1, srv1609279463=25, srv742743553=83, srv873551319=89, srv1588550243=23, srv575526193=75, srv881451098=90, srv1652935864=30, srv1840211266=45, srv153034324=22, srv2051788734=57, srv319629628=67, srv1787581600=41, srv749023457=84, srv947904493=95, srv1727294131=33, srv1648686423=29, srv1873131725=47, srv967729145=97, srv1236984426=12, srv1771335643=39, srv306083069=66, srv1996549164=54, srv945210971=94, srv609447077=77, srv1095082718=7, srv1185455147=10, srv1972966800=53, srv2120382996=59, srv325432250=69, srv1430771729=17, srv189909834=50, srv553105152=73, srv1052887025=3, srv1281205085=15, srv1514574662=21, srv776123721=86, srv1908447515=51, srv1044735449=2, srv1455230607=19, srv772329470=85, srv97714977=98, srv1693027323=31, srv709129020=81, srv998431403=99, srv1615999575=27, srv582194224=76, srv299178434=65, srv202385378=55, srv2124385648=61, srv444799157=71, srv1055374215=4, srv1732510198=35, srv1825060679=44, srv501177272=72, srv322545395=68, srv1731747001=34, srv562953736=74, srv268114086=64, srv161579350=26, srv933045416=92, srv2064493420=58, srv1150743170=8, srv417959332=70, srv1261063185=14, srv1821175562=43, srv949432644=96, srv237696716=63, srv1853867956=46, srv694235394=80, srv675508027=79, srv1441289698=18} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-15 19:53:28,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-15 19:53:28,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-15 19:53:28,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-15 19:53:28,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-15 19:53:28,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-15 19:53:28,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-15 19:53:28,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-15 19:53:28,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-15 19:53:28,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-15 19:53:28,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-15 19:53:28,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-15 19:53:28,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-15 19:53:28,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-15 19:53:28,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-15 19:53:28,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-15 19:53:28,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-15 19:53:28,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-15 19:53:28,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-15 19:53:28,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-15 19:53:28,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-15 19:53:28,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-15 19:53:28,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-15 19:53:28,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-15 19:53:28,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-15 19:53:28,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-15 19:53:28,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-15 19:53:28,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-15 19:53:28,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-15 19:53:28,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-15 19:53:28,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-15 19:53:28,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-15 19:53:28,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-15 19:53:28,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-15 19:53:28,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-15 19:53:28,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-15 19:53:28,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-15 19:53:28,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-15 19:53:28,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-15 19:53:28,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-15 19:53:28,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-15 19:53:28,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-15 19:53:28,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-15 19:53:28,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-15 19:53:28,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-15 19:53:28,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-15 19:53:28,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-15 19:53:28,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-15 19:53:28,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-15 19:53:28,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-15 19:53:28,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-15 19:53:28,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-15 19:53:28,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-15 19:53:28,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-15 19:53:28,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-15 19:53:28,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-15 19:53:28,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-15 19:53:28,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-15 19:53:28,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-15 19:53:28,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-15 19:53:28,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-15 19:53:28,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-15 19:53:28,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-15 19:53:28,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-15 19:53:28,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-15 19:53:28,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-15 19:53:28,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-15 19:53:28,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-15 19:53:28,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-15 19:53:28,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-15 19:53:28,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-15 19:53:28,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-15 19:53:28,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-15 19:53:28,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-15 19:53:28,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-15 19:53:28,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-15 19:53:28,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-15 19:53:28,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-15 19:53:28,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-15 19:53:28,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-15 19:53:28,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-15 19:53:28,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-15 19:53:28,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-15 19:53:28,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-15 19:53:28,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-15 19:53:28,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-15 19:53:28,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-15 19:53:28,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-15 19:53:28,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-15 19:53:28,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-15 19:53:28,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-15 19:53:28,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-15 19:53:28,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-15 19:53:28,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-15 19:53:28,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-15 19:53:28,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-15 19:53:28,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-15 19:53:28,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-15 19:53:28,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-15 19:53:28,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-15 19:53:28,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-15 19:53:28,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-15 19:53:29,120 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-15 19:53:29,120 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-15 19:53:29,121 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-15 19:53:29,736 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 948 ms to try 5000 different iterations.  Found a solution that moves 4704 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18126381521172444. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04049913052786327, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05783020154727872, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5226666666666666, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04049913052786327, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1765, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.243, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-15 19:53:29,756 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1185455147:82 , srv1514574662:82 , srv1877782356:82 , srv1041090471:83 , srv1236984426:83 , srv1430771729:83 , srv1052887025:84 , srv1253019227:84 , srv161579350:84 , srv1648686423:84 , srv1739223862:84 , srv1744666379:84 , srv1806580104:84 , srv1924069238:84 , srv1996549164:84 , srv2051788734:84 , srv2124385648:84 , srv237696716:84 , srv949432644:84 , srv1007986791:85 , srv1261063185:85 , srv1455230607:85 , srv1474899482:85 , srv1615999575:85 , srv1693027323:85 , srv1727294131:85 , srv1771335643:85 , srv1873131725:85 , srv202385378:85 , srv2120382996:85 , srv2131095941:85 , srv319629628:85 , srv322545395:85 , srv582194224:85 , srv709129020:85 , srv738531535:85 , srv749023457:85 , srv933045416:85 , srv97714977:85 , srv1055374215:86 , srv1071801527:86 , srv1095082718:86 , srv1179842772:86 , srv1320749546:86 , srv1588550243:86 , srv1595009264:86 , srv1609279463:86 , srv1726876634:86 , srv1731747001:86 , srv1732510198:86 , srv1784586942:86 , srv1787581600:86 , srv1821175562:86 , srv1892224385:86 , srv2122796101:86 , srv299178434:86 , srv325432250:86 , srv553105152:86 , srv694235394:86 , srv742743553:86 , srv776123721:86 , srv776638132:86 , srv787756481:86 , srv881451098:86 , srv945210971:86 , srv1091695394:87 , srv1150743170:87 , srv1281205085:87 , srv1648253456:87 , srv1972966800:87 , srv444799157:87 , srv675508027:87 , srv772329470:87 , srv939138896:87 , srv967729145:87 , srv1224647658:88 , srv1441289698:88 , srv1652935864:88 , srv1751166597:88 , srv1840211266:88 , srv501177272:88 , srv562953736:88 , srv873551319:88 , srv947904493:88 , srv1044735449:89 , srv153034324:89 , srv1908447515:89 , srv1825060679:90 , srv189909834:90 , srv2036368900:90 , srv306083069:90 , srv998431403:90 , srv417959332:91 , srv575526193:91 , srv90110108:91 , srv268114086:92 , srv1853867956:93 , srv609447077:93 , srv2064493420:97 , srv632053504:450 }
2023-10-15 19:53:29,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-15 19:53:29,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1806580104=42, srv1474899482=20, srv632053504=78, srv1877782356=48, srv90110108=91, srv1739223862=36, srv1924069238=52, srv1071801527=5, srv1253019227=13, srv1320749546=16, srv2131095941=62, srv1892224385=49, srv1595009264=24, srv1726876634=32, srv1744666379=37, srv1179842772=9, srv776638132=87, srv939138896=93, srv2036368900=56, srv1648253456=28, srv738531535=82, srv1091695394=6, srv1007986791=0, srv1224647658=11, srv1784586942=40, srv1751166597=38, srv787756481=88, srv2122796101=60, srv1041090471=1, srv1609279463=25, srv742743553=83, srv873551319=89, srv1588550243=23, srv575526193=75, srv881451098=90, srv1652935864=30, srv1840211266=45, srv153034324=22, srv2051788734=57, srv319629628=67, srv1787581600=41, srv749023457=84, srv947904493=95, srv1727294131=33, srv1648686423=29, srv1873131725=47, srv967729145=97, srv1236984426=12, srv1771335643=39, srv306083069=66, srv1996549164=54, srv945210971=94, srv609447077=77, srv1095082718=7, srv1185455147=10, srv1972966800=53, srv2120382996=59, srv325432250=69, srv1430771729=17, srv189909834=50, srv553105152=73, srv1052887025=3, srv1281205085=15, srv1514574662=21, srv776123721=86, srv1908447515=51, srv1044735449=2, srv1455230607=19, srv772329470=85, srv97714977=98, srv1693027323=31, srv709129020=81, srv998431403=99, srv1615999575=27, srv582194224=76, srv299178434=65, srv202385378=55, srv2124385648=61, srv444799157=71, srv1055374215=4, srv1732510198=35, srv1825060679=44, srv501177272=72, srv322545395=68, srv1731747001=34, srv562953736=74, srv268114086=64, srv161579350=26, srv933045416=92, srv2064493420=58, srv1150743170=8, srv417959332=70, srv1261063185=14, srv1821175562=43, srv949432644=96, srv237696716=63, srv1853867956=46, srv694235394=80, srv675508027=79, srv1441289698=18} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-15 19:53:29,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-15 19:53:29,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-15 19:53:29,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-15 19:53:29,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-15 19:53:29,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-15 19:53:29,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-15 19:53:29,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-15 19:53:29,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-15 19:53:29,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-15 19:53:29,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-15 19:53:29,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-15 19:53:29,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-15 19:53:29,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-15 19:53:29,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-15 19:53:29,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-15 19:53:29,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-15 19:53:29,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-15 19:53:29,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-15 19:53:29,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-15 19:53:29,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-15 19:53:29,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-15 19:53:29,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-15 19:53:29,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-15 19:53:29,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-15 19:53:29,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-15 19:53:29,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-15 19:53:29,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-15 19:53:29,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-15 19:53:29,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-15 19:53:29,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-15 19:53:29,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-15 19:53:29,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-15 19:53:29,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-15 19:53:29,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-15 19:53:29,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-15 19:53:29,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-15 19:53:29,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-15 19:53:29,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-15 19:53:29,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-15 19:53:29,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-15 19:53:29,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-15 19:53:29,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-15 19:53:29,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-15 19:53:29,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-15 19:53:29,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-15 19:53:29,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-15 19:53:29,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-15 19:53:29,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-15 19:53:29,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-15 19:53:29,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-15 19:53:29,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-15 19:53:29,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-15 19:53:29,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-15 19:53:29,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-15 19:53:29,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-15 19:53:29,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-15 19:53:29,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-15 19:53:29,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-15 19:53:29,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-15 19:53:29,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-15 19:53:29,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-15 19:53:29,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-15 19:53:29,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-15 19:53:29,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-15 19:53:29,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-15 19:53:29,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-15 19:53:29,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-15 19:53:29,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-15 19:53:29,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-15 19:53:29,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-15 19:53:29,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-15 19:53:29,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-15 19:53:29,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-15 19:53:29,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-15 19:53:29,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-15 19:53:29,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-15 19:53:29,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-15 19:53:29,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-15 19:53:29,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-15 19:53:29,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-15 19:53:29,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-15 19:53:29,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-15 19:53:29,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-15 19:53:29,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-15 19:53:29,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-15 19:53:29,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-15 19:53:29,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-15 19:53:29,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-15 19:53:29,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-15 19:53:29,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-15 19:53:29,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-15 19:53:29,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-15 19:53:29,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-15 19:53:29,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-15 19:53:29,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-15 19:53:29,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-15 19:53:29,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-15 19:53:29,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-15 19:53:29,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-15 19:53:29,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-15 19:53:29,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-15 19:53:29,951 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04049913052786327, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05783020154727872, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04049913052786327, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1765, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.243, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-15 19:53:29,952 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-15 19:53:29,952 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18123087709972405, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04049913052786327, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05783020154727872, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04049913052786327, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1765, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.243, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-15 19:53:30,272 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 441 ms to try 5000 different iterations.  Found a solution that moves 2669 regions; Going from a computed imbalance of 0.18123087709972405 to a new imbalance of 0.015639256374958294. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020262032076849477); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03349120862247865, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29655555555555557, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020262032076849477); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012583333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04491666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-15 19:53:30,285 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv2064493420:74 , srv609447077:77 , srv268114086:79 , srv189909834:80 , srv90110108:80 , srv1853867956:81 , srv575526193:81 , srv998431403:81 , srv1825060679:82 , srv2036368900:82 , srv306083069:82 , srv1044735449:83 , srv153034324:83 , srv417959332:83 , srv742743553:83 , srv947904493:83 , srv1091695394:84 , srv1224647658:84 , srv1441289698:84 , srv1615999575:84 , srv1751166597:84 , srv1908447515:84 , srv444799157:84 , srv562953736:84 , srv939138896:84 , srv967729145:84 , srv1150743170:85 , srv1588550243:85 , srv1648253456:85 , srv2122796101:85 , srv325432250:85 , srv553105152:85 , srv675508027:85 , srv709129020:85 , srv776638132:85 , srv873551319:85 , srv1055374215:86 , srv1179842772:86 , srv1320749546:86 , srv1455230607:86 , srv1652935864:86 , srv1726876634:86 , srv1732510198:86 , srv1784586942:86 , srv1787581600:86 , srv1840211266:86 , srv1873131725:86 , srv2120382996:86 , srv2124385648:86 , srv299178434:86 , srv322545395:86 , srv738531535:86 , srv772329470:86 , srv881451098:86 , srv933045416:86 , srv97714977:86 , srv1071801527:87 , srv1095082718:87 , srv1236984426:87 , srv1253019227:87 , srv1261063185:87 , srv1281205085:87 , srv1430771729:87 , srv1474899482:87 , srv1693027323:87 , srv1731747001:87 , srv1744666379:87 , srv1771335643:87 , srv1821175562:87 , srv1972966800:87 , srv202385378:87 , srv2131095941:87 , srv319629628:87 , srv501177272:87 , srv694235394:87 , srv787756481:87 , srv1052887025:88 , srv161579350:88 , srv1727294131:88 , srv1739223862:88 , srv1892224385:88 , srv1996549164:88 , srv2051788734:88 , srv749023457:88 , srv945210971:88 , srv949432644:88 , srv1595009264:89 , srv1609279463:89 , srv1648686423:89 , srv237696716:89 , srv582194224:89 , srv776123721:89 , srv1007986791:90 , srv1185455147:90 , srv1514574662:91 , srv1806580104:91 , srv1041090471:92 , srv1877782356:92 , srv1924069238:92 , srv632053504:504 }
2023-10-15 19:53:30,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-15 19:53:30,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1806580104=42, srv1474899482=20, srv632053504=78, srv1877782356=48, srv90110108=91, srv1739223862=36, srv1924069238=52, srv1071801527=5, srv1253019227=13, srv1320749546=16, srv2131095941=62, srv1892224385=49, srv1595009264=24, srv1726876634=32, srv1744666379=37, srv1179842772=9, srv776638132=87, srv939138896=93, srv2036368900=56, srv1648253456=28, srv738531535=82, srv1091695394=6, srv1007986791=0, srv1224647658=11, srv1784586942=40, srv1751166597=38, srv787756481=88, srv2122796101=60, srv1041090471=1, srv1609279463=25, srv742743553=83, srv873551319=89, srv1588550243=23, srv575526193=75, srv881451098=90, srv1652935864=30, srv1840211266=45, srv153034324=22, srv2051788734=57, srv319629628=67, srv1787581600=41, srv749023457=84, srv947904493=95, srv1727294131=33, srv1648686423=29, srv1873131725=47, srv967729145=97, srv1236984426=12, srv1771335643=39, srv306083069=66, srv1996549164=54, srv945210971=94, srv609447077=77, srv1095082718=7, srv1185455147=10, srv1972966800=53, srv2120382996=59, srv325432250=69, srv1430771729=17, srv189909834=50, srv553105152=73, srv1052887025=3, srv1281205085=15, srv1514574662=21, srv776123721=86, srv1908447515=51, srv1044735449=2, srv1455230607=19, srv772329470=85, srv97714977=98, srv1693027323=31, srv709129020=81, srv998431403=99, srv1615999575=27, srv582194224=76, srv299178434=65, srv202385378=55, srv2124385648=61, srv444799157=71, srv1055374215=4, srv1732510198=35, srv1825060679=44, srv501177272=72, srv322545395=68, srv1731747001=34, srv562953736=74, srv268114086=64, srv161579350=26, srv933045416=92, srv2064493420=58, srv1150743170=8, srv417959332=70, srv1261063185=14, srv1821175562=43, srv949432644=96, srv237696716=63, srv1853867956=46, srv694235394=80, srv675508027=79, srv1441289698=18} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-15 19:53:30,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-15 19:53:30,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-15 19:53:30,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-15 19:53:30,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-15 19:53:30,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-15 19:53:30,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-15 19:53:30,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-15 19:53:30,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-15 19:53:30,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-15 19:53:30,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-15 19:53:30,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-15 19:53:30,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-15 19:53:30,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-15 19:53:30,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-15 19:53:30,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-15 19:53:30,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-15 19:53:30,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-15 19:53:30,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-15 19:53:30,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-15 19:53:30,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-15 19:53:30,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-15 19:53:30,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-15 19:53:30,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-15 19:53:30,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-15 19:53:30,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-15 19:53:30,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-15 19:53:30,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-15 19:53:30,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-15 19:53:30,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-15 19:53:30,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-15 19:53:30,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-15 19:53:30,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-15 19:53:30,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-15 19:53:30,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-15 19:53:30,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-15 19:53:30,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-15 19:53:30,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-15 19:53:30,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-15 19:53:30,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-15 19:53:30,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-15 19:53:30,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-15 19:53:30,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-15 19:53:30,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-15 19:53:30,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-15 19:53:30,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-15 19:53:30,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-15 19:53:30,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-15 19:53:30,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-15 19:53:30,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-15 19:53:30,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-15 19:53:30,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-15 19:53:30,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-15 19:53:30,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-15 19:53:30,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-15 19:53:30,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-15 19:53:30,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-15 19:53:30,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-15 19:53:30,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-15 19:53:30,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-15 19:53:30,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-15 19:53:30,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-15 19:53:30,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-15 19:53:30,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-15 19:53:30,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-15 19:53:30,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-15 19:53:30,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-15 19:53:30,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-15 19:53:30,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-15 19:53:30,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-15 19:53:30,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-15 19:53:30,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-15 19:53:30,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-15 19:53:30,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-15 19:53:30,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-15 19:53:30,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-15 19:53:30,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-15 19:53:30,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-15 19:53:30,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-15 19:53:30,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-15 19:53:30,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-15 19:53:30,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-15 19:53:30,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-15 19:53:30,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-15 19:53:30,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-15 19:53:30,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-15 19:53:30,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-15 19:53:30,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-15 19:53:30,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-15 19:53:30,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-15 19:53:30,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-15 19:53:30,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-15 19:53:30,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-15 19:53:30,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-15 19:53:30,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-15 19:53:30,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-15 19:53:30,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-15 19:53:30,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-15 19:53:30,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-15 19:53:30,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-15 19:53:30,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-15 19:53:30,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-15 19:53:30,458 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020262032076849477); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03349120862247865, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020262032076849477); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012583333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04491666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-15 19:53:30,458 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-15 19:53:30,458 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.015620567637515901, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020262032076849477); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03349120862247865, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020262032076849477); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012583333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04491666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-15 19:53:30,869 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 523 ms to try 5000 different iterations.  Found a solution that moves 1030 regions; Going from a computed imbalance of 0.015620567637515901 to a new imbalance of 6.187245371987722E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004544332072404845); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007654139963827327); MoveCostFunction : (multiplier=7.0, imbalance=0.11444444444444445, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.004544332072404845); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0036666666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-15 19:53:30,871 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1007986791:82 , srv1595009264:82 , srv1609279463:82 , srv1806580104:82 , srv1924069238:82 , srv501177272:82 , srv1041090471:83 , srv1840211266:83 , srv1892224385:83 , srv417959332:83 , srv776123721:83 , srv1095082718:84 , srv1648686423:84 , srv1652935864:84 , srv1731747001:84 , srv1821175562:84 , srv1853867956:84 , srv1877782356:84 , srv1908447515:84 , srv1972966800:84 , srv237696716:84 , srv582194224:84 , srv749023457:84 , srv945210971:84 , srv1071801527:85 , srv1179842772:85 , srv1224647658:85 , srv1261063185:85 , srv1281205085:85 , srv1441289698:85 , srv1514574662:85 , srv161579350:85 , srv1693027323:85 , srv1726876634:85 , srv1727294131:85 , srv1732510198:85 , srv1751166597:85 , srv1787581600:85 , srv2036368900:85 , srv319629628:85 , srv322545395:85 , srv562953736:85 , srv575526193:85 , srv694235394:85 , srv772329470:85 , srv787756481:85 , srv873551319:85 , srv881451098:85 , srv947904493:85 , srv949432644:85 , srv1044735449:86 , srv1052887025:86 , srv1055374215:86 , srv1091695394:86 , srv1150743170:86 , srv1253019227:86 , srv1320749546:86 , srv153034324:86 , srv1648253456:86 , srv1739223862:86 , srv1744666379:86 , srv1771335643:86 , srv1784586942:86 , srv1996549164:86 , srv202385378:86 , srv2051788734:86 , srv2120382996:86 , srv2122796101:86 , srv268114086:86 , srv299178434:86 , srv306083069:86 , srv675508027:86 , srv967729145:86 , srv1185455147:87 , srv1236984426:87 , srv1455230607:87 , srv1474899482:87 , srv1588550243:87 , srv1825060679:87 , srv1873131725:87 , srv189909834:87 , srv2064493420:87 , srv2124385648:87 , srv2131095941:87 , srv325432250:87 , srv444799157:87 , srv553105152:87 , srv738531535:87 , srv776638132:87 , srv90110108:87 , srv933045416:87 , srv97714977:87 , srv998431403:87 , srv1430771729:88 , srv609447077:88 , srv709129020:88 , srv742743553:88 , srv939138896:88 , srv1615999575:89 , srv632053504:544 }
2023-10-15 19:53:30,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-15 19:53:30,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1806580104=42, srv1474899482=20, srv632053504=78, srv1877782356=48, srv90110108=91, srv1739223862=36, srv1924069238=52, srv1071801527=5, srv1253019227=13, srv1320749546=16, srv2131095941=62, srv1892224385=49, srv1595009264=24, srv1726876634=32, srv1744666379=37, srv1179842772=9, srv776638132=87, srv939138896=93, srv2036368900=56, srv1648253456=28, srv738531535=82, srv1091695394=6, srv1007986791=0, srv1224647658=11, srv1784586942=40, srv1751166597=38, srv787756481=88, srv2122796101=60, srv1041090471=1, srv1609279463=25, srv742743553=83, srv873551319=89, srv1588550243=23, srv575526193=75, srv881451098=90, srv1652935864=30, srv1840211266=45, srv153034324=22, srv2051788734=57, srv319629628=67, srv1787581600=41, srv749023457=84, srv947904493=95, srv1727294131=33, srv1648686423=29, srv1873131725=47, srv967729145=97, srv1236984426=12, srv1771335643=39, srv306083069=66, srv1996549164=54, srv945210971=94, srv609447077=77, srv1095082718=7, srv1185455147=10, srv1972966800=53, srv2120382996=59, srv325432250=69, srv1430771729=17, srv189909834=50, srv553105152=73, srv1052887025=3, srv1281205085=15, srv1514574662=21, srv776123721=86, srv1908447515=51, srv1044735449=2, srv1455230607=19, srv772329470=85, srv97714977=98, srv1693027323=31, srv709129020=81, srv998431403=99, srv1615999575=27, srv582194224=76, srv299178434=65, srv202385378=55, srv2124385648=61, srv444799157=71, srv1055374215=4, srv1732510198=35, srv1825060679=44, srv501177272=72, srv322545395=68, srv1731747001=34, srv562953736=74, srv268114086=64, srv161579350=26, srv933045416=92, srv2064493420=58, srv1150743170=8, srv417959332=70, srv1261063185=14, srv1821175562=43, srv949432644=96, srv237696716=63, srv1853867956=46, srv694235394=80, srv675508027=79, srv1441289698=18} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-15 19:53:30,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-15 19:53:30,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-15 19:53:30,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-15 19:53:30,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-15 19:53:30,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-15 19:53:30,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-15 19:53:30,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-15 19:53:30,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-15 19:53:30,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-15 19:53:30,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-15 19:53:30,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-15 19:53:30,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-15 19:53:30,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-15 19:53:30,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-15 19:53:30,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-15 19:53:30,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-15 19:53:30,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-15 19:53:30,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-15 19:53:30,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-15 19:53:30,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-15 19:53:30,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-15 19:53:30,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-15 19:53:30,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-15 19:53:30,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-15 19:53:30,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-15 19:53:30,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-15 19:53:30,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-15 19:53:30,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-15 19:53:30,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-15 19:53:30,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-15 19:53:30,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-15 19:53:30,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-15 19:53:30,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-15 19:53:30,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-15 19:53:30,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-15 19:53:30,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-15 19:53:30,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-15 19:53:30,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-15 19:53:30,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-15 19:53:30,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-15 19:53:30,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-15 19:53:30,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-15 19:53:30,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-15 19:53:30,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-15 19:53:30,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-15 19:53:30,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-15 19:53:30,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-15 19:53:30,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-15 19:53:30,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-15 19:53:30,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-15 19:53:30,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-15 19:53:30,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-15 19:53:30,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-15 19:53:30,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-15 19:53:30,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-15 19:53:30,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-15 19:53:30,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-15 19:53:30,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-15 19:53:30,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-15 19:53:30,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-15 19:53:30,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-15 19:53:30,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-15 19:53:30,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-15 19:53:30,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-15 19:53:30,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-15 19:53:30,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-15 19:53:30,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-15 19:53:30,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-15 19:53:30,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-15 19:53:30,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-15 19:53:30,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-15 19:53:30,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-15 19:53:30,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-15 19:53:30,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-15 19:53:30,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-15 19:53:30,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-15 19:53:30,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-15 19:53:30,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-15 19:53:30,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-15 19:53:30,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-15 19:53:30,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-15 19:53:30,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-15 19:53:30,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-15 19:53:30,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-15 19:53:30,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-15 19:53:30,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-15 19:53:30,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-15 19:53:30,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-15 19:53:30,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-15 19:53:30,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-15 19:53:30,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-15 19:53:30,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-15 19:53:30,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-15 19:53:30,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-15 19:53:30,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-15 19:53:30,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-15 19:53:30,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-15 19:53:30,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-15 19:53:30,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-15 19:53:30,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-15 19:53:30,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-15 19:53:31,039 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004544332072404845); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007654139963827327); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.004544332072404845); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0036666666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-15 19:53:31,039 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-15 19:53:31,039 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=6.115123230490283E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004544332072404845); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007654139963827327); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.004544332072404845); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0036666666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-15 19:53:31,319 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 404 ms to try 5000 different iterations.  Found a solution that moves 249 regions; Going from a computed imbalance of 6.115123230490283E-4 to a new imbalance of 2.000955882289364E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=1.5792645655971144E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0022222222222222222); MoveCostFunction : (multiplier=7.0, imbalance=0.027666666666666666, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.5792645655971144E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-15 19:53:31,320 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1185455147:83 , srv1281205085:83 , srv1474899482:83 , srv1825060679:83 , srv2131095941:83 , srv939138896:83 , srv1041090471:84 , srv1044735449:84 , srv1052887025:84 , srv1055374215:84 , srv1071801527:84 , srv1150743170:84 , srv1236984426:84 , srv1320749546:84 , srv1430771729:84 , srv1455230607:84 , srv1514574662:84 , srv153034324:84 , srv1588550243:84 , srv1615999575:84 , srv1648253456:84 , srv1652935864:84 , srv1727294131:84 , srv1739223862:84 , srv1771335643:84 , srv1784586942:84 , srv1853867956:84 , srv1873131725:84 , srv1877782356:84 , srv1924069238:84 , srv1996549164:84 , srv202385378:84 , srv2051788734:84 , srv2064493420:84 , srv299178434:84 , srv306083069:84 , srv325432250:84 , srv444799157:84 , srv553105152:84 , srv582194224:84 , srv609447077:84 , srv675508027:84 , srv694235394:84 , srv709129020:84 , srv738531535:84 , srv772329470:84 , srv776123721:84 , srv776638132:84 , srv787756481:84 , srv873551319:84 , srv90110108:84 , srv933045416:84 , srv945210971:84 , srv97714977:84 , srv998431403:84 , srv1007986791:85 , srv1091695394:85 , srv1095082718:85 , srv1179842772:85 , srv1224647658:85 , srv1253019227:85 , srv1261063185:85 , srv1441289698:85 , srv1595009264:85 , srv1609279463:85 , srv161579350:85 , srv1648686423:85 , srv1693027323:85 , srv1726876634:85 , srv1731747001:85 , srv1732510198:85 , srv1744666379:85 , srv1751166597:85 , srv1787581600:85 , srv1806580104:85 , srv1821175562:85 , srv1840211266:85 , srv1892224385:85 , srv189909834:85 , srv1908447515:85 , srv1972966800:85 , srv2036368900:85 , srv2120382996:85 , srv2122796101:85 , srv2124385648:85 , srv237696716:85 , srv268114086:85 , srv319629628:85 , srv417959332:85 , srv501177272:85 , srv562953736:85 , srv575526193:85 , srv742743553:85 , srv749023457:85 , srv881451098:85 , srv949432644:85 , srv967729145:85 , srv322545395:86 , srv947904493:86 , srv632053504:644 }
2023-10-15 19:53:31,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-15 19:53:31,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1806580104=42, srv1474899482=20, srv632053504=78, srv1877782356=48, srv90110108=91, srv1739223862=36, srv1924069238=52, srv1071801527=5, srv1253019227=13, srv1320749546=16, srv2131095941=62, srv1892224385=49, srv1595009264=24, srv1726876634=32, srv1744666379=37, srv1179842772=9, srv776638132=87, srv939138896=93, srv2036368900=56, srv1648253456=28, srv738531535=82, srv1091695394=6, srv1007986791=0, srv1224647658=11, srv1784586942=40, srv1751166597=38, srv787756481=88, srv2122796101=60, srv1041090471=1, srv1609279463=25, srv742743553=83, srv873551319=89, srv1588550243=23, srv575526193=75, srv881451098=90, srv1652935864=30, srv1840211266=45, srv153034324=22, srv2051788734=57, srv319629628=67, srv1787581600=41, srv749023457=84, srv947904493=95, srv1727294131=33, srv1648686423=29, srv1873131725=47, srv967729145=97, srv1236984426=12, srv1771335643=39, srv306083069=66, srv1996549164=54, srv945210971=94, srv609447077=77, srv1095082718=7, srv1185455147=10, srv1972966800=53, srv2120382996=59, srv325432250=69, srv1430771729=17, srv189909834=50, srv553105152=73, srv1052887025=3, srv1281205085=15, srv1514574662=21, srv776123721=86, srv1908447515=51, srv1044735449=2, srv1455230607=19, srv772329470=85, srv97714977=98, srv1693027323=31, srv709129020=81, srv998431403=99, srv1615999575=27, srv582194224=76, srv299178434=65, srv202385378=55, srv2124385648=61, srv444799157=71, srv1055374215=4, srv1732510198=35, srv1825060679=44, srv501177272=72, srv322545395=68, srv1731747001=34, srv562953736=74, srv268114086=64, srv161579350=26, srv933045416=92, srv2064493420=58, srv1150743170=8, srv417959332=70, srv1261063185=14, srv1821175562=43, srv949432644=96, srv237696716=63, srv1853867956=46, srv694235394=80, srv675508027=79, srv1441289698=18} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-15 19:53:31,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-15 19:53:31,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-15 19:53:31,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-15 19:53:31,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-15 19:53:31,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-15 19:53:31,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-15 19:53:31,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-15 19:53:31,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-15 19:53:31,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-15 19:53:31,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-15 19:53:31,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-15 19:53:31,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-15 19:53:31,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-15 19:53:31,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-15 19:53:31,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-15 19:53:31,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-15 19:53:31,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-15 19:53:31,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-15 19:53:31,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-15 19:53:31,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-15 19:53:31,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-15 19:53:31,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-15 19:53:31,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-15 19:53:31,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-15 19:53:31,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-15 19:53:31,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-15 19:53:31,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-15 19:53:31,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-15 19:53:31,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-15 19:53:31,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-15 19:53:31,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-15 19:53:31,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-15 19:53:31,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-15 19:53:31,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-15 19:53:31,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-15 19:53:31,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-15 19:53:31,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-15 19:53:31,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-15 19:53:31,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-15 19:53:31,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-15 19:53:31,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-15 19:53:31,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-15 19:53:31,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-15 19:53:31,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-15 19:53:31,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-15 19:53:31,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-15 19:53:31,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-15 19:53:31,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-15 19:53:31,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-15 19:53:31,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-15 19:53:31,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-15 19:53:31,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-15 19:53:31,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-15 19:53:31,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-15 19:53:31,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-15 19:53:31,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-15 19:53:31,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-15 19:53:31,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-15 19:53:31,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-15 19:53:31,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-15 19:53:31,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-15 19:53:31,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-15 19:53:31,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-15 19:53:31,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-15 19:53:31,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-15 19:53:31,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-15 19:53:31,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-15 19:53:31,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-15 19:53:31,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-15 19:53:31,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-15 19:53:31,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-15 19:53:31,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-15 19:53:31,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-15 19:53:31,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-15 19:53:31,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-15 19:53:31,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-15 19:53:31,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-15 19:53:31,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-15 19:53:31,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-15 19:53:31,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-15 19:53:31,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-15 19:53:31,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-15 19:53:31,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-15 19:53:31,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-15 19:53:31,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-15 19:53:31,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-15 19:53:31,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-15 19:53:31,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-15 19:53:31,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-15 19:53:31,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-15 19:53:31,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-15 19:53:31,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-15 19:53:31,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-15 19:53:31,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-15 19:53:31,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-15 19:53:31,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-15 19:53:31,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-15 19:53:31,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-15 19:53:31,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-15 19:53:31,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-15 19:53:31,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-15 19:53:31,459 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.8266023557567183E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=1.5792645655971144E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0022222222222222222); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.5792645655971144E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-15 19:53:31,459 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1185455147:83 , srv1281205085:83 , srv1474899482:83 , srv1825060679:83 , srv2131095941:83 , srv939138896:83 , srv1041090471:84 , srv1044735449:84 , srv1052887025:84 , srv1055374215:84 , srv1071801527:84 , srv1150743170:84 , srv1236984426:84 , srv1320749546:84 , srv1430771729:84 , srv1455230607:84 , srv1514574662:84 , srv153034324:84 , srv1588550243:84 , srv1615999575:84 , srv1648253456:84 , srv1652935864:84 , srv1727294131:84 , srv1739223862:84 , srv1771335643:84 , srv1784586942:84 , srv1853867956:84 , srv1873131725:84 , srv1877782356:84 , srv1924069238:84 , srv1996549164:84 , srv202385378:84 , srv2051788734:84 , srv2064493420:84 , srv299178434:84 , srv306083069:84 , srv325432250:84 , srv444799157:84 , srv553105152:84 , srv582194224:84 , srv609447077:84 , srv675508027:84 , srv694235394:84 , srv709129020:84 , srv738531535:84 , srv772329470:84 , srv776123721:84 , srv776638132:84 , srv787756481:84 , srv873551319:84 , srv90110108:84 , srv933045416:84 , srv945210971:84 , srv97714977:84 , srv998431403:84 , srv1007986791:85 , srv1091695394:85 , srv1095082718:85 , srv1179842772:85 , srv1224647658:85 , srv1253019227:85 , srv1261063185:85 , srv1441289698:85 , srv1595009264:85 , srv1609279463:85 , srv161579350:85 , srv1648686423:85 , srv1693027323:85 , srv1726876634:85 , srv1731747001:85 , srv1732510198:85 , srv1744666379:85 , srv1751166597:85 , srv1787581600:85 , srv1806580104:85 , srv1821175562:85 , srv1840211266:85 , srv1892224385:85 , srv189909834:85 , srv1908447515:85 , srv1972966800:85 , srv2036368900:85 , srv2120382996:85 , srv2122796101:85 , srv2124385648:85 , srv237696716:85 , srv268114086:85 , srv319629628:85 , srv417959332:85 , srv501177272:85 , srv562953736:85 , srv575526193:85 , srv742743553:85 , srv749023457:85 , srv881451098:85 , srv949432644:85 , srv967729145:85 , srv322545395:86 , srv947904493:86 , srv632053504:644 }
2023-10-15 19:53:31,491 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=323 (was 323), ProcessCount=170 (was 170), AvailableMemoryMB=5877 (was 6648)
2023-10-15 19:53:31,499 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=323, ProcessCount=170, AvailableMemoryMB=5877
2023-10-15 19:53:31,500 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-10-15 19:53:31,500 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-10-15 19:53:31,513 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv102352692:84 , srv1063663640:84 , srv1162135010:84 , srv1201574272:84 , srv1221375262:84 , srv1237846897:84 , srv1454384376:84 , srv1466511096:84 , srv1575952389:84 , srv1635421881:84 , srv1656512442:84 , srv1669937236:84 , srv167965260:84 , srv1844808813:84 , srv1882459354:84 , srv1948539041:84 , srv1988272694:84 , srv2084796273:84 , srv213880061:84 , srv396762292:84 , srv397174559:84 , srv419926784:84 , srv469005372:84 , srv469834256:84 , srv488852555:84 , srv568160586:84 , srv610702388:84 , srv651116573:84 , srv972192566:84 , srv1827924746:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-10-15 19:53:31,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-15 19:53:31,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1844808813=14, srv2084796273=18, srv972192566=29, srv102352692=0, srv396762292=20, srv1656512442=10, srv610702388=27, srv1454384376=6, srv1466511096=7, srv1221375262=4, srv469005372=23, srv469834256=24, srv1635421881=9, srv1162135010=2, srv1063663640=1, srv167965260=12, srv488852555=25, srv1882459354=15, srv651116573=28, srv397174559=21, srv1948539041=16, srv1827924746=13, srv213880061=19, srv1669937236=11, srv1575952389=8, srv419926784=22, srv568160586=26, srv1201574272=3, srv1237846897=5, srv1988272694=17} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-15 19:53:31,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-15 19:53:31,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-15 19:53:31,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-15 19:53:31,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-15 19:53:31,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-15 19:53:31,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-15 19:53:31,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-15 19:53:31,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-15 19:53:31,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-15 19:53:31,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-15 19:53:31,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-15 19:53:31,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-15 19:53:31,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-15 19:53:31,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-15 19:53:31,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-15 19:53:31,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-15 19:53:31,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-15 19:53:31,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-15 19:53:31,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-15 19:53:31,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-15 19:53:31,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-15 19:53:31,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-15 19:53:31,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-15 19:53:31,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-15 19:53:31,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-15 19:53:31,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-15 19:53:31,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-15 19:53:31,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-15 19:53:31,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-15 19:53:31,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-15 19:53:31,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-10-15 19:53:31,556 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1264025611146051, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-15 19:53:31,557 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912591333606928, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1264025611146051, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-10-15 19:55:31,535 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5679744 different iterations.  Found a solution that moves 1855 regions; Going from a computed imbalance of 0.9912591333606928 to a new imbalance of 4.329662539733032E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.687037037037037, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-15 19:55:31,542 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv102352692:90 , srv1063663640:90 , srv1162135010:90 , srv1201574272:90 , srv1221375262:90 , srv1237846897:90 , srv1454384376:90 , srv1466511096:90 , srv1575952389:90 , srv1635421881:90 , srv1656512442:90 , srv1669937236:90 , srv167965260:90 , srv1827924746:90 , srv1844808813:90 , srv1882459354:90 , srv1948539041:90 , srv1988272694:90 , srv2084796273:90 , srv213880061:90 , srv396762292:90 , srv397174559:90 , srv419926784:90 , srv469005372:90 , srv469834256:90 , srv488852555:90 , srv568160586:90 , srv610702388:90 , srv651116573:90 , srv972192566:90 }
2023-10-15 19:55:31,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-15 19:55:31,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1844808813=14, srv2084796273=18, srv972192566=29, srv102352692=0, srv396762292=20, srv1656512442=10, srv610702388=27, srv1454384376=6, srv1466511096=7, srv1221375262=4, srv469005372=23, srv469834256=24, srv1635421881=9, srv1162135010=2, srv1063663640=1, srv167965260=12, srv488852555=25, srv1882459354=15, srv651116573=28, srv397174559=21, srv1948539041=16, srv1827924746=13, srv213880061=19, srv1669937236=11, srv1575952389=8, srv419926784=22, srv568160586=26, srv1201574272=3, srv1237846897=5, srv1988272694=17} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-15 19:55:31,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-15 19:55:31,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-15 19:55:31,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-15 19:55:31,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-15 19:55:31,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-15 19:55:31,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-15 19:55:31,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-15 19:55:31,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-15 19:55:31,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-15 19:55:31,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-15 19:55:31,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-15 19:55:31,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-15 19:55:31,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-15 19:55:31,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-15 19:55:31,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-15 19:55:31,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-15 19:55:31,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-15 19:55:31,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-15 19:55:31,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-15 19:55:31,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-15 19:55:31,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-15 19:55:31,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-15 19:55:31,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-15 19:55:31,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-15 19:55:31,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-15 19:55:31,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-15 19:55:31,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-15 19:55:31,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-15 19:55:31,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-15 19:55:31,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-15 19:55:31,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-10-15 19:55:31,561 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-15 19:55:31,561 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv102352692:90 , srv1063663640:90 , srv1162135010:90 , srv1201574272:90 , srv1221375262:90 , srv1237846897:90 , srv1454384376:90 , srv1466511096:90 , srv1575952389:90 , srv1635421881:90 , srv1656512442:90 , srv1669937236:90 , srv167965260:90 , srv1827924746:90 , srv1844808813:90 , srv1882459354:90 , srv1948539041:90 , srv1988272694:90 , srv2084796273:90 , srv213880061:90 , srv396762292:90 , srv397174559:90 , srv419926784:90 , srv469005372:90 , srv469834256:90 , srv488852555:90 , srv568160586:90 , srv610702388:90 , srv651116573:90 , srv972192566:90 }
2023-10-15 19:55:31,573 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=164 (was 323), ProcessCount=170 (was 170), AvailableMemoryMB=6306 (was 5877) - AvailableMemoryMB LEAK? -