2023-04-14 21:55:41,462 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-04-14 21:55:41,748 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-04-14 21:55:41,852 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-04-14 21:55:41,911 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-14 21:55:41,918 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-04-14 21:55:41,931 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=423, ProcessCount=169, AvailableMemoryMB=7110
2023-04-14 21:55:41,934 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-14 21:55:41,934 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-04-14 21:55:42,048 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1047353555:84 , srv1082255177:84 , srv1093310659:84 , srv1103166001:84 , srv1126221221:84 , srv1142034693:84 , srv1228463777:84 , srv1228985122:84 , srv1240318563:84 , srv126278282:84 , srv1262884194:84 , srv1287855272:84 , srv1328286766:84 , srv133458106:84 , srv135976466:84 , srv1369652509:84 , srv1385782306:84 , srv1391776627:84 , srv1395478465:84 , srv1399343853:84 , srv1404978944:84 , srv1411649790:84 , srv1438938105:84 , srv1450037641:84 , srv1450094354:84 , srv1502696902:84 , srv1520864479:84 , srv153541569:84 , srv1561263928:84 , srv1591230599:84 , srv1614478428:84 , srv1617476063:84 , srv161918251:84 , srv1628792273:84 , srv1646925391:84 , srv1659858099:84 , srv1717345983:84 , srv1717420247:84 , srv1758854586:84 , srv17924163:84 , srv1875613244:84 , srv1892366923:84 , srv19049408:84 , srv1937433137:84 , srv1941737540:84 , srv1973914868:84 , srv1984239360:84 , srv2026636085:84 , srv2028348381:84 , srv2065632824:84 , srv2104563083:84 , srv2115533530:84 , srv2145028727:84 , srv230208712:84 , srv268954958:84 , srv301007190:84 , srv304451366:84 , srv305559828:84 , srv338302831:84 , srv369184639:84 , srv372890764:84 , srv388096286:84 , srv414421510:84 , srv42404836:84 , srv427349412:84 , srv478167519:84 , srv486721261:84 , srv497794157:84 , srv527516207:84 , srv580954432:84 , srv591137745:84 , srv602054064:84 , srv617221534:84 , srv635051012:84 , srv639533562:84 , srv666922290:84 , srv6702051:84 , srv684207282:84 , srv702809404:84 , srv729265503:84 , srv762439983:84 , srv830711686:84 , srv84712109:84 , srv854314677:84 , srv854707715:84 , srv872957406:84 , srv88653226:84 , srv905791281:84 , srv907201686:84 , srv91704527:84 , srv919629210:84 , srv922606501:84 , srv942439270:84 , srv943103467:84 , srv943294987:84 , srv962177957:84 , srv967746059:84 , srv990544637:84 , srv990845430:84 , srv487226138:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-04-14 21:55:42,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-14 21:55:42,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv126278282=9, srv369184639=59, srv1228985122=7, srv1404978944=20, srv1502696902=25, srv2026636085=47, srv990544637=98, srv1391776627=17, srv1228463777=6, srv414421510=62, srv1520864479=26, srv1591230599=29, srv1614478428=30, srv617221534=73, srv639533562=75, srv1450037641=23, srv1240318563=8, srv133458106=13, srv91704527=90, srv427349412=64, srv1717345983=36, srv487226138=67, srv1892366923=41, srv2145028727=52, srv153541569=27, srv702809404=79, srv1328286766=12, srv580954432=70, srv943103467=94, srv304451366=56, srv635051012=74, srv830711686=82, srv1369652509=15, srv527516207=69, srv268954958=54, srv1973914868=45, srv872957406=86, srv1646925391=34, srv1399343853=19, srv497794157=68, srv1438938105=22, srv602054064=72, srv684207282=78, srv854314677=84, srv1628792273=33, srv1093310659=2, srv1262884194=10, srv42404836=63, srv1411649790=21, srv907201686=89, srv1875613244=40, srv486721261=66, srv372890764=60, srv943294987=95, srv922606501=92, srv1395478465=18, srv905791281=88, srv17924163=39, srv2115533530=51, srv135976466=14, srv1984239360=46, srv230208712=53, srv1937433137=43, srv854707715=85, srv1617476063=31, srv2028348381=48, srv1103166001=3, srv762439983=81, srv388096286=61, srv478167519=65, srv19049408=42, srv990845430=99, srv1659858099=35, srv1717420247=37, srv1126221221=4, srv942439270=93, srv1561263928=28, srv1758854586=38, srv1287855272=11, srv666922290=76, srv1450094354=24, srv729265503=80, srv88653226=87, srv1082255177=1, srv305559828=57, srv967746059=97, srv591137745=71, srv301007190=55, srv338302831=58, srv2104563083=50, srv919629210=91, srv1941737540=44, srv84712109=83, srv161918251=32, srv962177957=96, srv1142034693=5, srv6702051=77, srv1385782306=16, srv2065632824=49, srv1047353555=0} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-14 21:55:42,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-14 21:55:42,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-14 21:55:42,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-14 21:55:42,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-14 21:55:42,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-14 21:55:42,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-14 21:55:42,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-14 21:55:42,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-14 21:55:42,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-14 21:55:42,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-14 21:55:42,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-14 21:55:42,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-14 21:55:42,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-14 21:55:42,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-14 21:55:42,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-14 21:55:42,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-14 21:55:42,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-14 21:55:42,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-14 21:55:42,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-14 21:55:42,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-14 21:55:42,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-14 21:55:42,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-14 21:55:42,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-14 21:55:42,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-14 21:55:42,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-14 21:55:42,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-14 21:55:42,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-14 21:55:42,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-14 21:55:42,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-14 21:55:42,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-14 21:55:42,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-14 21:55:42,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-14 21:55:42,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-14 21:55:42,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-14 21:55:42,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-14 21:55:42,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-14 21:55:42,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-14 21:55:42,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-14 21:55:42,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-14 21:55:42,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-14 21:55:42,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-14 21:55:42,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-14 21:55:42,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-14 21:55:42,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-14 21:55:42,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-14 21:55:42,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-14 21:55:42,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-14 21:55:42,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-14 21:55:42,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-14 21:55:42,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-14 21:55:42,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-14 21:55:42,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-14 21:55:42,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-14 21:55:42,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-14 21:55:42,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-14 21:55:42,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-14 21:55:42,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-14 21:55:42,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-14 21:55:42,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-14 21:55:42,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-14 21:55:42,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-14 21:55:42,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-14 21:55:42,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-14 21:55:42,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-14 21:55:42,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-14 21:55:42,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-14 21:55:42,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-14 21:55:42,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-14 21:55:42,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-14 21:55:42,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-14 21:55:42,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-14 21:55:42,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-14 21:55:42,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-14 21:55:42,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-14 21:55:42,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-14 21:55:42,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-14 21:55:42,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-14 21:55:42,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-14 21:55:42,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-14 21:55:42,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-14 21:55:42,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-14 21:55:42,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-14 21:55:42,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-14 21:55:42,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-14 21:55:42,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-14 21:55:42,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-14 21:55:42,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-14 21:55:42,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-14 21:55:42,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-14 21:55:42,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-14 21:55:42,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-14 21:55:42,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-14 21:55:42,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-14 21:55:42,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-14 21:55:42,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-14 21:55:42,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-14 21:55:42,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-14 21:55:42,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-14 21:55:42,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-14 21:55:42,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-14 21:55:42,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-14 21:55:42,346 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-14 21:55:42,347 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-14 21:55:42,347 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-14 21:55:42,632 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 447 ms to try 5000 different iterations.  Found a solution that moves 4718 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17647391854927483. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0375649248833796, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.055186653992610496, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5242222222222223, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0375649248833796, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17183333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23675, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-14 21:55:42,641 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1093310659:83 , srv1395478465:83 , srv1411649790:83 , srv1875613244:83 , srv602054064:83 , srv1103166001:84 , srv1142034693:84 , srv1502696902:84 , srv1892366923:84 , srv2028348381:84 , srv2104563083:84 , srv478167519:84 , srv639533562:84 , srv729265503:84 , srv922606501:84 , srv1126221221:85 , srv1228463777:85 , srv135976466:85 , srv1369652509:85 , srv1438938105:85 , srv1561263928:85 , srv161918251:85 , srv1937433137:85 , srv2026636085:85 , srv372890764:85 , srv414421510:85 , srv497794157:85 , srv635051012:85 , srv702809404:85 , srv762439983:85 , srv854707715:85 , srv962177957:85 , srv967746059:85 , srv990544637:85 , srv1047353555:86 , srv1082255177:86 , srv1228985122:86 , srv1240318563:86 , srv133458106:86 , srv1617476063:86 , srv17924163:86 , srv1941737540:86 , srv2145028727:86 , srv230208712:86 , srv338302831:86 , srv617221534:86 , srv830711686:86 , srv872957406:86 , srv88653226:86 , srv943294987:86 , srv1328286766:87 , srv1404978944:87 , srv1646925391:87 , srv1758854586:87 , srv19049408:87 , srv1984239360:87 , srv2115533530:87 , srv268954958:87 , srv301007190:87 , srv304451366:87 , srv305559828:87 , srv6702051:87 , srv84712109:87 , srv854314677:87 , srv91704527:87 , srv919629210:87 , srv942439270:87 , srv990845430:87 , srv1262884194:88 , srv1287855272:88 , srv1385782306:88 , srv1450094354:88 , srv1520864479:88 , srv1591230599:88 , srv2065632824:88 , srv369184639:88 , srv388096286:88 , srv427349412:88 , srv486721261:88 , srv580954432:88 , srv666922290:88 , srv684207282:88 , srv905791281:88 , srv907201686:88 , srv126278282:89 , srv1399343853:89 , srv1614478428:89 , srv1628792273:89 , srv527516207:89 , srv1391776627:90 , srv153541569:90 , srv1659858099:90 , srv1717345983:90 , srv1973914868:90 , srv1450037641:91 , srv943103467:91 , srv1717420247:92 , srv591137745:92 , srv42404836:95 , srv487226138:424 }
2023-04-14 21:55:42,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-14 21:55:42,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv126278282=9, srv369184639=59, srv1228985122=7, srv1404978944=20, srv1502696902=25, srv2026636085=47, srv990544637=98, srv1391776627=17, srv1228463777=6, srv414421510=62, srv1520864479=26, srv1591230599=29, srv1614478428=30, srv617221534=73, srv639533562=75, srv1450037641=23, srv1240318563=8, srv133458106=13, srv91704527=90, srv427349412=64, srv1717345983=36, srv487226138=67, srv1892366923=41, srv2145028727=52, srv153541569=27, srv702809404=79, srv1328286766=12, srv580954432=70, srv943103467=94, srv304451366=56, srv635051012=74, srv830711686=82, srv1369652509=15, srv527516207=69, srv268954958=54, srv1973914868=45, srv872957406=86, srv1646925391=34, srv1399343853=19, srv497794157=68, srv1438938105=22, srv602054064=72, srv684207282=78, srv854314677=84, srv1628792273=33, srv1093310659=2, srv1262884194=10, srv42404836=63, srv1411649790=21, srv907201686=89, srv1875613244=40, srv486721261=66, srv372890764=60, srv943294987=95, srv922606501=92, srv1395478465=18, srv905791281=88, srv17924163=39, srv2115533530=51, srv135976466=14, srv1984239360=46, srv230208712=53, srv1937433137=43, srv854707715=85, srv1617476063=31, srv2028348381=48, srv1103166001=3, srv762439983=81, srv388096286=61, srv478167519=65, srv19049408=42, srv990845430=99, srv1659858099=35, srv1717420247=37, srv1126221221=4, srv942439270=93, srv1561263928=28, srv1758854586=38, srv1287855272=11, srv666922290=76, srv1450094354=24, srv729265503=80, srv88653226=87, srv1082255177=1, srv305559828=57, srv967746059=97, srv591137745=71, srv301007190=55, srv338302831=58, srv2104563083=50, srv919629210=91, srv1941737540=44, srv84712109=83, srv161918251=32, srv962177957=96, srv1142034693=5, srv6702051=77, srv1385782306=16, srv2065632824=49, srv1047353555=0} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-14 21:55:42,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-14 21:55:42,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-14 21:55:42,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-14 21:55:42,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-14 21:55:42,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-14 21:55:42,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-14 21:55:42,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-14 21:55:42,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-14 21:55:42,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-14 21:55:42,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-14 21:55:42,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-14 21:55:42,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-14 21:55:42,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-14 21:55:42,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-14 21:55:42,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-14 21:55:42,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-14 21:55:42,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-14 21:55:42,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-14 21:55:42,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-14 21:55:42,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-14 21:55:42,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-14 21:55:42,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-14 21:55:42,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-14 21:55:42,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-14 21:55:42,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-14 21:55:42,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-14 21:55:42,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-14 21:55:42,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-14 21:55:42,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-14 21:55:42,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-14 21:55:42,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-14 21:55:42,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-14 21:55:42,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-14 21:55:42,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-14 21:55:42,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-14 21:55:42,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-14 21:55:42,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-14 21:55:42,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-14 21:55:42,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-14 21:55:42,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-14 21:55:42,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-14 21:55:42,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-14 21:55:42,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-14 21:55:42,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-14 21:55:42,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-14 21:55:42,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-14 21:55:42,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-14 21:55:42,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-14 21:55:42,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-14 21:55:42,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-14 21:55:42,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-14 21:55:42,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-14 21:55:42,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-14 21:55:42,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-14 21:55:42,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-14 21:55:42,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-14 21:55:42,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-14 21:55:42,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-14 21:55:42,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-14 21:55:42,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-14 21:55:42,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-14 21:55:42,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-14 21:55:42,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-14 21:55:42,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-14 21:55:42,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-14 21:55:42,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-14 21:55:42,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-14 21:55:42,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-14 21:55:42,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-14 21:55:42,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-14 21:55:42,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-14 21:55:42,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-14 21:55:42,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-14 21:55:42,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-14 21:55:42,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-14 21:55:42,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-14 21:55:42,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-14 21:55:42,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-14 21:55:42,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-14 21:55:42,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-14 21:55:42,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-14 21:55:42,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-14 21:55:42,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-14 21:55:42,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-14 21:55:42,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-14 21:55:42,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-14 21:55:42,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-14 21:55:42,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-14 21:55:42,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-14 21:55:42,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-14 21:55:42,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-14 21:55:42,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-14 21:55:42,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-14 21:55:42,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-14 21:55:42,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-14 21:55:42,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-14 21:55:42,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-14 21:55:42,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-14 21:55:42,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-14 21:55:42,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-14 21:55:42,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-14 21:55:42,798 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0375649248833796, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.055186653992610496, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0375649248833796, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17183333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23675, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-14 21:55:42,798 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-14 21:55:42,798 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17644088240717923, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0375649248833796, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.055186653992610496, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0375649248833796, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17183333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23675, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-14 21:55:43,055 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 365 ms to try 5000 different iterations.  Found a solution that moves 2646 regions; Going from a computed imbalance of 0.17644088240717923 to a new imbalance of 0.014776312943195577. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02160593231639763); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03204934467079101, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.294, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02160593231639763); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.041166666666666664, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-14 21:55:43,064 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv42404836:75 , srv1717420247:79 , srv153541569:80 , srv905791281:80 , srv126278282:81 , srv1391776627:81 , srv591137745:81 , srv943103467:81 , srv1450037641:82 , srv388096286:82 , srv527516207:82 , srv684207282:82 , srv1262884194:83 , srv1287855272:83 , srv1399343853:83 , srv1520864479:83 , srv1717345983:83 , srv1973914868:83 , srv1984239360:83 , srv666922290:83 , srv1385782306:84 , srv1614478428:84 , srv1628792273:84 , srv1659858099:84 , srv1758854586:84 , srv301007190:84 , srv369184639:84 , srv486721261:84 , srv854314677:84 , srv907201686:84 , srv91704527:84 , srv943294987:84 , srv1228985122:85 , srv1450094354:85 , srv1617476063:85 , srv1941737540:85 , srv427349412:85 , srv478167519:85 , srv617221534:85 , srv6702051:85 , srv942439270:85 , srv1142034693:86 , srv1240318563:86 , srv1328286766:86 , srv1404978944:86 , srv1561263928:86 , srv161918251:86 , srv1646925391:86 , srv17924163:86 , srv2028348381:86 , srv2065632824:86 , srv268954958:86 , srv304451366:86 , srv338302831:86 , srv580954432:86 , srv639533562:86 , srv762439983:86 , srv830711686:86 , srv919629210:86 , srv990845430:86 , srv1047353555:87 , srv1126221221:87 , srv1228463777:87 , srv133458106:87 , srv135976466:87 , srv1369652509:87 , srv1438938105:87 , srv1502696902:87 , srv1591230599:87 , srv1875613244:87 , srv1937433137:87 , srv2115533530:87 , srv2145028727:87 , srv230208712:87 , srv305559828:87 , srv372890764:87 , srv635051012:87 , srv702809404:87 , srv854707715:87 , srv872957406:87 , srv967746059:87 , srv1395478465:88 , srv1892366923:88 , srv19049408:88 , srv2026636085:88 , srv2104563083:88 , srv497794157:88 , srv602054064:88 , srv84712109:88 , srv88653226:88 , srv922606501:88 , srv962177957:88 , srv990544637:88 , srv1093310659:89 , srv1411649790:89 , srv414421510:89 , srv1082255177:90 , srv1103166001:90 , srv729265503:91 , srv487226138:542 }
2023-04-14 21:55:43,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-14 21:55:43,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv126278282=9, srv369184639=59, srv1228985122=7, srv1404978944=20, srv1502696902=25, srv2026636085=47, srv990544637=98, srv1391776627=17, srv1228463777=6, srv414421510=62, srv1520864479=26, srv1591230599=29, srv1614478428=30, srv617221534=73, srv639533562=75, srv1450037641=23, srv1240318563=8, srv133458106=13, srv91704527=90, srv427349412=64, srv1717345983=36, srv487226138=67, srv1892366923=41, srv2145028727=52, srv153541569=27, srv702809404=79, srv1328286766=12, srv580954432=70, srv943103467=94, srv304451366=56, srv635051012=74, srv830711686=82, srv1369652509=15, srv527516207=69, srv268954958=54, srv1973914868=45, srv872957406=86, srv1646925391=34, srv1399343853=19, srv497794157=68, srv1438938105=22, srv602054064=72, srv684207282=78, srv854314677=84, srv1628792273=33, srv1093310659=2, srv1262884194=10, srv42404836=63, srv1411649790=21, srv907201686=89, srv1875613244=40, srv486721261=66, srv372890764=60, srv943294987=95, srv922606501=92, srv1395478465=18, srv905791281=88, srv17924163=39, srv2115533530=51, srv135976466=14, srv1984239360=46, srv230208712=53, srv1937433137=43, srv854707715=85, srv1617476063=31, srv2028348381=48, srv1103166001=3, srv762439983=81, srv388096286=61, srv478167519=65, srv19049408=42, srv990845430=99, srv1659858099=35, srv1717420247=37, srv1126221221=4, srv942439270=93, srv1561263928=28, srv1758854586=38, srv1287855272=11, srv666922290=76, srv1450094354=24, srv729265503=80, srv88653226=87, srv1082255177=1, srv305559828=57, srv967746059=97, srv591137745=71, srv301007190=55, srv338302831=58, srv2104563083=50, srv919629210=91, srv1941737540=44, srv84712109=83, srv161918251=32, srv962177957=96, srv1142034693=5, srv6702051=77, srv1385782306=16, srv2065632824=49, srv1047353555=0} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-14 21:55:43,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-14 21:55:43,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-14 21:55:43,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-14 21:55:43,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-14 21:55:43,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-14 21:55:43,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-14 21:55:43,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-14 21:55:43,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-14 21:55:43,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-14 21:55:43,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-14 21:55:43,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-14 21:55:43,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-14 21:55:43,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-14 21:55:43,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-14 21:55:43,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-14 21:55:43,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-14 21:55:43,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-14 21:55:43,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-14 21:55:43,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-14 21:55:43,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-14 21:55:43,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-14 21:55:43,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-14 21:55:43,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-14 21:55:43,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-14 21:55:43,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-14 21:55:43,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-14 21:55:43,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-14 21:55:43,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-14 21:55:43,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-14 21:55:43,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-14 21:55:43,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-14 21:55:43,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-14 21:55:43,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-14 21:55:43,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-14 21:55:43,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-14 21:55:43,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-14 21:55:43,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-14 21:55:43,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-14 21:55:43,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-14 21:55:43,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-14 21:55:43,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-14 21:55:43,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-14 21:55:43,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-14 21:55:43,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-14 21:55:43,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-14 21:55:43,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-14 21:55:43,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-14 21:55:43,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-14 21:55:43,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-14 21:55:43,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-14 21:55:43,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-14 21:55:43,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-14 21:55:43,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-14 21:55:43,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-14 21:55:43,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-14 21:55:43,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-14 21:55:43,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-14 21:55:43,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-14 21:55:43,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-14 21:55:43,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-14 21:55:43,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-14 21:55:43,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-14 21:55:43,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-14 21:55:43,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-14 21:55:43,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-14 21:55:43,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-14 21:55:43,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-14 21:55:43,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-14 21:55:43,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-14 21:55:43,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-14 21:55:43,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-14 21:55:43,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-14 21:55:43,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-14 21:55:43,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-14 21:55:43,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-14 21:55:43,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-14 21:55:43,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-14 21:55:43,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-14 21:55:43,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-14 21:55:43,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-14 21:55:43,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-14 21:55:43,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-14 21:55:43,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-14 21:55:43,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-14 21:55:43,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-14 21:55:43,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-14 21:55:43,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-14 21:55:43,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-14 21:55:43,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-14 21:55:43,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-14 21:55:43,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-14 21:55:43,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-14 21:55:43,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-14 21:55:43,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-14 21:55:43,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-14 21:55:43,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-14 21:55:43,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-14 21:55:43,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-14 21:55:43,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-14 21:55:43,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-14 21:55:43,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-14 21:55:43,200 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02160593231639763); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03204934467079101, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02160593231639763); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.041166666666666664, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-14 21:55:43,200 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-14 21:55:43,200 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.01475778525519536, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02160593231639763); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03204934467079101, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02160593231639763); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.041166666666666664, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-14 21:55:43,495 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 397 ms to try 5000 different iterations.  Found a solution that moves 960 regions; Going from a computed imbalance of 0.01475778525519536 to a new imbalance of 5.760726594577433E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0063328312164356885); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006374049520922626); MoveCostFunction : (multiplier=7.0, imbalance=0.10666666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0063328312164356885); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0031666666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-14 21:55:43,499 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1103166001:82 , srv19049408:82 , srv2115533530:82 , srv1082255177:83 , srv1591230599:83 , srv1614478428:83 , srv1659858099:83 , srv2065632824:83 , srv305559828:83 , srv580954432:83 , srv729265503:83 , srv84712109:83 , srv1328286766:84 , srv133458106:84 , srv1450037641:84 , srv1717345983:84 , srv17924163:84 , srv1937433137:84 , srv2145028727:84 , srv230208712:84 , srv268954958:84 , srv427349412:84 , srv497794157:84 , srv591137745:84 , srv702809404:84 , srv872957406:84 , srv88653226:84 , srv942439270:84 , srv1047353555:85 , srv1093310659:85 , srv1126221221:85 , srv1228463777:85 , srv1369652509:85 , srv1404978944:85 , srv1411649790:85 , srv1450094354:85 , srv1561263928:85 , srv1628792273:85 , srv1646925391:85 , srv1973914868:85 , srv2026636085:85 , srv2104563083:85 , srv304451366:85 , srv338302831:85 , srv372890764:85 , srv414421510:85 , srv635051012:85 , srv6702051:85 , srv919629210:85 , srv922606501:85 , srv962177957:85 , srv990544637:85 , srv990845430:85 , srv1228985122:86 , srv1240318563:86 , srv1262884194:86 , srv1287855272:86 , srv135976466:86 , srv1385782306:86 , srv1391776627:86 , srv1399343853:86 , srv1438938105:86 , srv1617476063:86 , srv161918251:86 , srv1717420247:86 , srv1892366923:86 , srv369184639:86 , srv486721261:86 , srv762439983:86 , srv830711686:86 , srv854314677:86 , srv854707715:86 , srv943103467:86 , srv967746059:86 , srv1142034693:87 , srv126278282:87 , srv1395478465:87 , srv1502696902:87 , srv1520864479:87 , srv1758854586:87 , srv1941737540:87 , srv1984239360:87 , srv2028348381:87 , srv301007190:87 , srv388096286:87 , srv527516207:87 , srv617221534:87 , srv639533562:87 , srv666922290:87 , srv907201686:87 , srv91704527:87 , srv943294987:87 , srv153541569:88 , srv1875613244:88 , srv42404836:88 , srv602054064:88 , srv684207282:88 , srv478167519:89 , srv905791281:89 , srv487226138:548 }
2023-04-14 21:55:43,503 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-14 21:55:43,504 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv126278282=9, srv369184639=59, srv1228985122=7, srv1404978944=20, srv1502696902=25, srv2026636085=47, srv990544637=98, srv1391776627=17, srv1228463777=6, srv414421510=62, srv1520864479=26, srv1591230599=29, srv1614478428=30, srv617221534=73, srv639533562=75, srv1450037641=23, srv1240318563=8, srv133458106=13, srv91704527=90, srv427349412=64, srv1717345983=36, srv487226138=67, srv1892366923=41, srv2145028727=52, srv153541569=27, srv702809404=79, srv1328286766=12, srv580954432=70, srv943103467=94, srv304451366=56, srv635051012=74, srv830711686=82, srv1369652509=15, srv527516207=69, srv268954958=54, srv1973914868=45, srv872957406=86, srv1646925391=34, srv1399343853=19, srv497794157=68, srv1438938105=22, srv602054064=72, srv684207282=78, srv854314677=84, srv1628792273=33, srv1093310659=2, srv1262884194=10, srv42404836=63, srv1411649790=21, srv907201686=89, srv1875613244=40, srv486721261=66, srv372890764=60, srv943294987=95, srv922606501=92, srv1395478465=18, srv905791281=88, srv17924163=39, srv2115533530=51, srv135976466=14, srv1984239360=46, srv230208712=53, srv1937433137=43, srv854707715=85, srv1617476063=31, srv2028348381=48, srv1103166001=3, srv762439983=81, srv388096286=61, srv478167519=65, srv19049408=42, srv990845430=99, srv1659858099=35, srv1717420247=37, srv1126221221=4, srv942439270=93, srv1561263928=28, srv1758854586=38, srv1287855272=11, srv666922290=76, srv1450094354=24, srv729265503=80, srv88653226=87, srv1082255177=1, srv305559828=57, srv967746059=97, srv591137745=71, srv301007190=55, srv338302831=58, srv2104563083=50, srv919629210=91, srv1941737540=44, srv84712109=83, srv161918251=32, srv962177957=96, srv1142034693=5, srv6702051=77, srv1385782306=16, srv2065632824=49, srv1047353555=0} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-14 21:55:43,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-14 21:55:43,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-14 21:55:43,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-14 21:55:43,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-14 21:55:43,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-14 21:55:43,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-14 21:55:43,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-14 21:55:43,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-14 21:55:43,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-14 21:55:43,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-14 21:55:43,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-14 21:55:43,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-14 21:55:43,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-14 21:55:43,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-14 21:55:43,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-14 21:55:43,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-14 21:55:43,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-14 21:55:43,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-14 21:55:43,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-14 21:55:43,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-14 21:55:43,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-14 21:55:43,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-14 21:55:43,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-14 21:55:43,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-14 21:55:43,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-14 21:55:43,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-14 21:55:43,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-14 21:55:43,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-14 21:55:43,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-14 21:55:43,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-14 21:55:43,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-14 21:55:43,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-14 21:55:43,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-14 21:55:43,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-14 21:55:43,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-14 21:55:43,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-14 21:55:43,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-14 21:55:43,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-14 21:55:43,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-14 21:55:43,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-14 21:55:43,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-14 21:55:43,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-14 21:55:43,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-14 21:55:43,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-14 21:55:43,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-14 21:55:43,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-14 21:55:43,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-14 21:55:43,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-14 21:55:43,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-14 21:55:43,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-14 21:55:43,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-14 21:55:43,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-14 21:55:43,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-14 21:55:43,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-14 21:55:43,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-14 21:55:43,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-14 21:55:43,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-14 21:55:43,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-14 21:55:43,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-14 21:55:43,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-14 21:55:43,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-14 21:55:43,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-14 21:55:43,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-14 21:55:43,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-14 21:55:43,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-14 21:55:43,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-14 21:55:43,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-14 21:55:43,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-14 21:55:43,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-14 21:55:43,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-14 21:55:43,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-14 21:55:43,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-14 21:55:43,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-14 21:55:43,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-14 21:55:43,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-14 21:55:43,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-14 21:55:43,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-14 21:55:43,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-14 21:55:43,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-14 21:55:43,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-14 21:55:43,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-14 21:55:43,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-14 21:55:43,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-14 21:55:43,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-14 21:55:43,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-14 21:55:43,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-14 21:55:43,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-14 21:55:43,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-14 21:55:43,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-14 21:55:43,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-14 21:55:43,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-14 21:55:43,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-14 21:55:43,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-14 21:55:43,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-14 21:55:43,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-14 21:55:43,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-14 21:55:43,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-14 21:55:43,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-14 21:55:43,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-14 21:55:43,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-14 21:55:43,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-14 21:55:43,653 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0063328312164356885); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006374049520922626); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0063328312164356885); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0031666666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-14 21:55:43,653 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-14 21:55:43,653 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=5.693505957841954E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0063328312164356885); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006374049520922626); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0063328312164356885); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0031666666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-14 21:55:43,934 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 384 ms to try 5000 different iterations.  Found a solution that moves 210 regions; Going from a computed imbalance of 5.693505957841954E-4 to a new imbalance of 2.5249160921756428E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.5313429231318235E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0015713484026367722); MoveCostFunction : (multiplier=7.0, imbalance=0.023333333333333334); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.5313429231318235E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-14 21:55:43,935 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1082255177:83 , srv153541569:83 , srv414421510:83 , srv602054064:83 , srv907201686:83 , srv919629210:83 , srv1047353555:84 , srv1240318563:84 , srv135976466:84 , srv1385782306:84 , srv1395478465:84 , srv1399343853:84 , srv1404978944:84 , srv1438938105:84 , srv1450094354:84 , srv1502696902:84 , srv1520864479:84 , srv1591230599:84 , srv1617476063:84 , srv1628792273:84 , srv1646925391:84 , srv1758854586:84 , srv1875613244:84 , srv1892366923:84 , srv1941737540:84 , srv1973914868:84 , srv2026636085:84 , srv301007190:84 , srv304451366:84 , srv369184639:84 , srv42404836:84 , srv486721261:84 , srv527516207:84 , srv617221534:84 , srv666922290:84 , srv684207282:84 , srv729265503:84 , srv830711686:84 , srv84712109:84 , srv854707715:84 , srv88653226:84 , srv91704527:84 , srv922606501:84 , srv943103467:84 , srv962177957:84 , srv990544637:84 , srv990845430:84 , srv1093310659:85 , srv1103166001:85 , srv1126221221:85 , srv1142034693:85 , srv1228463777:85 , srv1228985122:85 , srv126278282:85 , srv1262884194:85 , srv1287855272:85 , srv1328286766:85 , srv1369652509:85 , srv1391776627:85 , srv1411649790:85 , srv1450037641:85 , srv161918251:85 , srv1659858099:85 , srv1717345983:85 , srv19049408:85 , srv1984239360:85 , srv2028348381:85 , srv2065632824:85 , srv2104563083:85 , srv2145028727:85 , srv230208712:85 , srv268954958:85 , srv305559828:85 , srv338302831:85 , srv372890764:85 , srv388096286:85 , srv427349412:85 , srv478167519:85 , srv580954432:85 , srv591137745:85 , srv635051012:85 , srv639533562:85 , srv6702051:85 , srv762439983:85 , srv854314677:85 , srv872957406:85 , srv905791281:85 , srv943294987:85 , srv967746059:85 , srv133458106:86 , srv1561263928:86 , srv1614478428:86 , srv1717420247:86 , srv17924163:86 , srv1937433137:86 , srv2115533530:86 , srv497794157:86 , srv702809404:86 , srv942439270:86 , srv487226138:628 }
2023-04-14 21:55:43,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-14 21:55:43,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv126278282=9, srv369184639=59, srv1228985122=7, srv1404978944=20, srv1502696902=25, srv2026636085=47, srv990544637=98, srv1391776627=17, srv1228463777=6, srv414421510=62, srv1520864479=26, srv1591230599=29, srv1614478428=30, srv617221534=73, srv639533562=75, srv1450037641=23, srv1240318563=8, srv133458106=13, srv91704527=90, srv427349412=64, srv1717345983=36, srv487226138=67, srv1892366923=41, srv2145028727=52, srv153541569=27, srv702809404=79, srv1328286766=12, srv580954432=70, srv943103467=94, srv304451366=56, srv635051012=74, srv830711686=82, srv1369652509=15, srv527516207=69, srv268954958=54, srv1973914868=45, srv872957406=86, srv1646925391=34, srv1399343853=19, srv497794157=68, srv1438938105=22, srv602054064=72, srv684207282=78, srv854314677=84, srv1628792273=33, srv1093310659=2, srv1262884194=10, srv42404836=63, srv1411649790=21, srv907201686=89, srv1875613244=40, srv486721261=66, srv372890764=60, srv943294987=95, srv922606501=92, srv1395478465=18, srv905791281=88, srv17924163=39, srv2115533530=51, srv135976466=14, srv1984239360=46, srv230208712=53, srv1937433137=43, srv854707715=85, srv1617476063=31, srv2028348381=48, srv1103166001=3, srv762439983=81, srv388096286=61, srv478167519=65, srv19049408=42, srv990845430=99, srv1659858099=35, srv1717420247=37, srv1126221221=4, srv942439270=93, srv1561263928=28, srv1758854586=38, srv1287855272=11, srv666922290=76, srv1450094354=24, srv729265503=80, srv88653226=87, srv1082255177=1, srv305559828=57, srv967746059=97, srv591137745=71, srv301007190=55, srv338302831=58, srv2104563083=50, srv919629210=91, srv1941737540=44, srv84712109=83, srv161918251=32, srv962177957=96, srv1142034693=5, srv6702051=77, srv1385782306=16, srv2065632824=49, srv1047353555=0} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-14 21:55:43,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-14 21:55:43,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-14 21:55:43,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-14 21:55:43,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-14 21:55:43,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-14 21:55:43,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-14 21:55:43,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-14 21:55:43,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-14 21:55:43,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-14 21:55:43,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-14 21:55:43,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-14 21:55:43,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-14 21:55:43,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-14 21:55:43,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-14 21:55:43,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-14 21:55:43,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-14 21:55:43,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-14 21:55:43,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-14 21:55:43,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-14 21:55:43,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-14 21:55:43,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-14 21:55:43,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-14 21:55:43,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-14 21:55:43,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-14 21:55:43,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-14 21:55:43,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-14 21:55:43,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-14 21:55:43,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-14 21:55:43,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-14 21:55:43,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-14 21:55:43,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-14 21:55:43,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-14 21:55:43,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-14 21:55:43,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-14 21:55:43,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-14 21:55:43,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-14 21:55:43,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-14 21:55:43,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-14 21:55:43,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-14 21:55:43,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-14 21:55:43,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-14 21:55:43,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-14 21:55:43,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-14 21:55:43,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-14 21:55:43,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-14 21:55:43,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-14 21:55:43,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-14 21:55:43,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-14 21:55:43,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-14 21:55:43,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-14 21:55:43,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-14 21:55:43,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-14 21:55:43,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-14 21:55:43,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-14 21:55:43,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-14 21:55:43,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-14 21:55:43,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-14 21:55:43,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-14 21:55:43,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-14 21:55:43,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-14 21:55:43,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-14 21:55:43,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-14 21:55:43,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-14 21:55:43,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-14 21:55:43,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-14 21:55:43,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-14 21:55:43,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-14 21:55:43,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-14 21:55:43,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-14 21:55:43,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-14 21:55:43,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-14 21:55:43,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-14 21:55:43,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-14 21:55:43,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-14 21:55:43,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-14 21:55:43,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-14 21:55:43,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-14 21:55:43,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-14 21:55:43,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-14 21:55:43,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-14 21:55:43,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-14 21:55:43,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-14 21:55:43,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-14 21:55:43,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-14 21:55:43,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-14 21:55:43,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-14 21:55:43,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-14 21:55:43,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-14 21:55:43,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-14 21:55:43,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-14 21:55:43,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-14 21:55:43,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-14 21:55:43,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-14 21:55:43,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-14 21:55:43,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-14 21:55:43,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-14 21:55:43,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-14 21:55:43,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-14 21:55:43,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-14 21:55:43,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-14 21:55:43,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-14 21:55:44,061 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.5313429231318235E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0015713484026367722); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.5313429231318235E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-14 21:55:44,061 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-14 21:55:44,062 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=2.377870949316785E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.5313429231318235E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0015713484026367722); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.5313429231318235E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-14 21:55:44,352 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 385 ms to try 5000 different iterations.  Found a solution that moves 21 regions; Going from a computed imbalance of 2.377870949316785E-5 to a new imbalance of 5.789538136385531E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0012535023884839248); MoveCostFunction : (multiplier=7.0, imbalance=0.0023333333333333335); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-14 21:55:44,352 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv133458106:83 , srv1717420247:83 , srv478167519:83 , srv497794157:83 , srv967746059:83 , srv1047353555:84 , srv1082255177:84 , srv1093310659:84 , srv1126221221:84 , srv1142034693:84 , srv1228463777:84 , srv1228985122:84 , srv1240318563:84 , srv126278282:84 , srv1262884194:84 , srv1287855272:84 , srv1328286766:84 , srv135976466:84 , srv1369652509:84 , srv1385782306:84 , srv1391776627:84 , srv1395478465:84 , srv1399343853:84 , srv1404978944:84 , srv1411649790:84 , srv1438938105:84 , srv1450037641:84 , srv1450094354:84 , srv1502696902:84 , srv1520864479:84 , srv1561263928:84 , srv1591230599:84 , srv1614478428:84 , srv161918251:84 , srv1628792273:84 , srv1646925391:84 , srv1659858099:84 , srv1717345983:84 , srv1758854586:84 , srv17924163:84 , srv1875613244:84 , srv1892366923:84 , srv19049408:84 , srv1937433137:84 , srv1941737540:84 , srv1973914868:84 , srv1984239360:84 , srv2026636085:84 , srv2028348381:84 , srv2065632824:84 , srv2104563083:84 , srv2115533530:84 , srv2145028727:84 , srv230208712:84 , srv268954958:84 , srv301007190:84 , srv304451366:84 , srv305559828:84 , srv338302831:84 , srv369184639:84 , srv372890764:84 , srv388096286:84 , srv414421510:84 , srv42404836:84 , srv427349412:84 , srv486721261:84 , srv527516207:84 , srv580954432:84 , srv591137745:84 , srv602054064:84 , srv617221534:84 , srv635051012:84 , srv639533562:84 , srv666922290:84 , srv6702051:84 , srv684207282:84 , srv702809404:84 , srv729265503:84 , srv762439983:84 , srv830711686:84 , srv84712109:84 , srv854314677:84 , srv854707715:84 , srv872957406:84 , srv88653226:84 , srv905791281:84 , srv907201686:84 , srv91704527:84 , srv942439270:84 , srv943103467:84 , srv943294987:84 , srv962177957:84 , srv990544637:84 , srv990845430:84 , srv1103166001:85 , srv153541569:85 , srv1617476063:85 , srv919629210:85 , srv922606501:85 , srv487226138:684 }
2023-04-14 21:55:44,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-14 21:55:44,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv126278282=9, srv369184639=59, srv1228985122=7, srv1404978944=20, srv1502696902=25, srv2026636085=47, srv990544637=98, srv1391776627=17, srv1228463777=6, srv414421510=62, srv1520864479=26, srv1591230599=29, srv1614478428=30, srv617221534=73, srv639533562=75, srv1450037641=23, srv1240318563=8, srv133458106=13, srv91704527=90, srv427349412=64, srv1717345983=36, srv487226138=67, srv1892366923=41, srv2145028727=52, srv153541569=27, srv702809404=79, srv1328286766=12, srv580954432=70, srv943103467=94, srv304451366=56, srv635051012=74, srv830711686=82, srv1369652509=15, srv527516207=69, srv268954958=54, srv1973914868=45, srv872957406=86, srv1646925391=34, srv1399343853=19, srv497794157=68, srv1438938105=22, srv602054064=72, srv684207282=78, srv854314677=84, srv1628792273=33, srv1093310659=2, srv1262884194=10, srv42404836=63, srv1411649790=21, srv907201686=89, srv1875613244=40, srv486721261=66, srv372890764=60, srv943294987=95, srv922606501=92, srv1395478465=18, srv905791281=88, srv17924163=39, srv2115533530=51, srv135976466=14, srv1984239360=46, srv230208712=53, srv1937433137=43, srv854707715=85, srv1617476063=31, srv2028348381=48, srv1103166001=3, srv762439983=81, srv388096286=61, srv478167519=65, srv19049408=42, srv990845430=99, srv1659858099=35, srv1717420247=37, srv1126221221=4, srv942439270=93, srv1561263928=28, srv1758854586=38, srv1287855272=11, srv666922290=76, srv1450094354=24, srv729265503=80, srv88653226=87, srv1082255177=1, srv305559828=57, srv967746059=97, srv591137745=71, srv301007190=55, srv338302831=58, srv2104563083=50, srv919629210=91, srv1941737540=44, srv84712109=83, srv161918251=32, srv962177957=96, srv1142034693=5, srv6702051=77, srv1385782306=16, srv2065632824=49, srv1047353555=0} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-14 21:55:44,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-14 21:55:44,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-14 21:55:44,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-14 21:55:44,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-14 21:55:44,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-14 21:55:44,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-14 21:55:44,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-14 21:55:44,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-14 21:55:44,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-14 21:55:44,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-14 21:55:44,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-14 21:55:44,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-14 21:55:44,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-14 21:55:44,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-14 21:55:44,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-14 21:55:44,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-14 21:55:44,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-14 21:55:44,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-14 21:55:44,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-14 21:55:44,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-14 21:55:44,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-14 21:55:44,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-14 21:55:44,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-14 21:55:44,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-14 21:55:44,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-14 21:55:44,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-14 21:55:44,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-14 21:55:44,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-14 21:55:44,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-14 21:55:44,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-14 21:55:44,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-14 21:55:44,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-14 21:55:44,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-14 21:55:44,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-14 21:55:44,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-14 21:55:44,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-14 21:55:44,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-14 21:55:44,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-14 21:55:44,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-14 21:55:44,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-14 21:55:44,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-14 21:55:44,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-14 21:55:44,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-14 21:55:44,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-14 21:55:44,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-14 21:55:44,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-14 21:55:44,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-14 21:55:44,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-14 21:55:44,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-14 21:55:44,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-14 21:55:44,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-14 21:55:44,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-14 21:55:44,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-14 21:55:44,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-14 21:55:44,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-14 21:55:44,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-14 21:55:44,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-14 21:55:44,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-14 21:55:44,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-14 21:55:44,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-14 21:55:44,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-14 21:55:44,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-14 21:55:44,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-14 21:55:44,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-14 21:55:44,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-14 21:55:44,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-14 21:55:44,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-14 21:55:44,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-14 21:55:44,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-14 21:55:44,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-14 21:55:44,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-14 21:55:44,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-14 21:55:44,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-14 21:55:44,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-14 21:55:44,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-14 21:55:44,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-14 21:55:44,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-14 21:55:44,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-14 21:55:44,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-14 21:55:44,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-14 21:55:44,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-14 21:55:44,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-14 21:55:44,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-14 21:55:44,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-14 21:55:44,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-14 21:55:44,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-14 21:55:44,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-14 21:55:44,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-14 21:55:44,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-14 21:55:44,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-14 21:55:44,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-14 21:55:44,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-14 21:55:44,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-14 21:55:44,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-14 21:55:44,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-14 21:55:44,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-14 21:55:44,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-14 21:55:44,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-14 21:55:44,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-14 21:55:44,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-14 21:55:44,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-14 21:55:44,535 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=5.642492993526674E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0012535023884839248); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-14 21:55:44,536 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv133458106:83 , srv1717420247:83 , srv478167519:83 , srv497794157:83 , srv967746059:83 , srv1047353555:84 , srv1082255177:84 , srv1093310659:84 , srv1126221221:84 , srv1142034693:84 , srv1228463777:84 , srv1228985122:84 , srv1240318563:84 , srv126278282:84 , srv1262884194:84 , srv1287855272:84 , srv1328286766:84 , srv135976466:84 , srv1369652509:84 , srv1385782306:84 , srv1391776627:84 , srv1395478465:84 , srv1399343853:84 , srv1404978944:84 , srv1411649790:84 , srv1438938105:84 , srv1450037641:84 , srv1450094354:84 , srv1502696902:84 , srv1520864479:84 , srv1561263928:84 , srv1591230599:84 , srv1614478428:84 , srv161918251:84 , srv1628792273:84 , srv1646925391:84 , srv1659858099:84 , srv1717345983:84 , srv1758854586:84 , srv17924163:84 , srv1875613244:84 , srv1892366923:84 , srv19049408:84 , srv1937433137:84 , srv1941737540:84 , srv1973914868:84 , srv1984239360:84 , srv2026636085:84 , srv2028348381:84 , srv2065632824:84 , srv2104563083:84 , srv2115533530:84 , srv2145028727:84 , srv230208712:84 , srv268954958:84 , srv301007190:84 , srv304451366:84 , srv305559828:84 , srv338302831:84 , srv369184639:84 , srv372890764:84 , srv388096286:84 , srv414421510:84 , srv42404836:84 , srv427349412:84 , srv486721261:84 , srv527516207:84 , srv580954432:84 , srv591137745:84 , srv602054064:84 , srv617221534:84 , srv635051012:84 , srv639533562:84 , srv666922290:84 , srv6702051:84 , srv684207282:84 , srv702809404:84 , srv729265503:84 , srv762439983:84 , srv830711686:84 , srv84712109:84 , srv854314677:84 , srv854707715:84 , srv872957406:84 , srv88653226:84 , srv905791281:84 , srv907201686:84 , srv91704527:84 , srv942439270:84 , srv943103467:84 , srv943294987:84 , srv962177957:84 , srv990544637:84 , srv990845430:84 , srv1103166001:85 , srv153541569:85 , srv1617476063:85 , srv919629210:85 , srv922606501:85 , srv487226138:684 }
2023-04-14 21:55:44,571 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=478 (was 423) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 169), AvailableMemoryMB=6606 (was 7110)
2023-04-14 21:55:44,582 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=478, ProcessCount=169, AvailableMemoryMB=6606
2023-04-14 21:55:44,583 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-14 21:55:44,583 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-04-14 21:55:44,598 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1007385017:84 , srv1052944281:84 , srv1057833127:84 , srv1168417272:84 , srv1323693174:84 , srv1347849854:84 , srv1571493607:84 , srv1639098071:84 , srv1761300056:84 , srv1769640147:84 , srv181474240:84 , srv182725658:84 , srv1848088716:84 , srv1859236641:84 , srv1963374216:84 , srv1965018439:84 , srv2011371813:84 , srv2014094588:84 , srv2044975645:84 , srv248651286:84 , srv301558743:84 , srv34170254:84 , srv381372942:84 , srv41970827:84 , srv560369459:84 , srv563257499:84 , srv590119527:84 , srv648293983:84 , srv982659110:84 , srv112344151:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-04-14 21:55:44,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-14 21:55:44,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2044975645=19, srv1859236641=14, srv563257499=26, srv1057833127=2, srv34170254=22, srv1965018439=16, srv112344151=3, srv1052944281=1, srv1963374216=15, srv1323693174=5, srv590119527=27, srv1848088716=13, srv560369459=25, srv181474240=11, srv1347849854=6, srv301558743=21, srv2011371813=17, srv648293983=28, srv1571493607=7, srv1761300056=9, srv1168417272=4, srv1769640147=10, srv248651286=20, srv381372942=23, srv1007385017=0, srv2014094588=18, srv982659110=29, srv41970827=24, srv182725658=12, srv1639098071=8} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-14 21:55:44,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-14 21:55:44,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-14 21:55:44,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-14 21:55:44,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-14 21:55:44,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-14 21:55:44,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-14 21:55:44,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-14 21:55:44,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-14 21:55:44,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-14 21:55:44,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-14 21:55:44,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-14 21:55:44,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-14 21:55:44,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-14 21:55:44,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-14 21:55:44,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-14 21:55:44,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-14 21:55:44,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-14 21:55:44,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-14 21:55:44,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-14 21:55:44,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-14 21:55:44,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-14 21:55:44,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-14 21:55:44,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-14 21:55:44,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-14 21:55:44,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-14 21:55:44,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-14 21:55:44,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-14 21:55:44,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-14 21:55:44,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-14 21:55:44,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-14 21:55:44,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-04-14 21:55:44,643 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1164646151091613, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-14 21:55:44,643 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912560019463569, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1164646151091613, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-04-14 21:57:44,622 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5607792 different iterations.  Found a solution that moves 1866 regions; Going from a computed imbalance of 0.9912560019463569 to a new imbalance of 4.3553370884861656E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6911111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-14 21:57:44,629 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1007385017:90 , srv1052944281:90 , srv1057833127:90 , srv112344151:90 , srv1168417272:90 , srv1323693174:90 , srv1347849854:90 , srv1571493607:90 , srv1639098071:90 , srv1761300056:90 , srv1769640147:90 , srv181474240:90 , srv182725658:90 , srv1848088716:90 , srv1859236641:90 , srv1963374216:90 , srv1965018439:90 , srv2011371813:90 , srv2014094588:90 , srv2044975645:90 , srv248651286:90 , srv301558743:90 , srv34170254:90 , srv381372942:90 , srv41970827:90 , srv560369459:90 , srv563257499:90 , srv590119527:90 , srv648293983:90 , srv982659110:90 }
2023-04-14 21:57:44,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-14 21:57:44,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2044975645=19, srv1859236641=14, srv563257499=26, srv1057833127=2, srv34170254=22, srv1965018439=16, srv112344151=3, srv1052944281=1, srv1963374216=15, srv1323693174=5, srv590119527=27, srv1848088716=13, srv560369459=25, srv181474240=11, srv1347849854=6, srv301558743=21, srv2011371813=17, srv648293983=28, srv1571493607=7, srv1761300056=9, srv1168417272=4, srv1769640147=10, srv248651286=20, srv381372942=23, srv1007385017=0, srv2014094588=18, srv982659110=29, srv41970827=24, srv182725658=12, srv1639098071=8} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-14 21:57:44,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-14 21:57:44,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-14 21:57:44,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-14 21:57:44,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-14 21:57:44,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-14 21:57:44,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-14 21:57:44,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-14 21:57:44,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-14 21:57:44,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-14 21:57:44,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-14 21:57:44,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-14 21:57:44,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-14 21:57:44,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-14 21:57:44,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-14 21:57:44,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-14 21:57:44,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-14 21:57:44,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-14 21:57:44,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-14 21:57:44,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-14 21:57:44,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-14 21:57:44,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-14 21:57:44,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-14 21:57:44,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-14 21:57:44,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-14 21:57:44,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-14 21:57:44,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-14 21:57:44,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-14 21:57:44,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-14 21:57:44,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-14 21:57:44,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-14 21:57:44,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-04-14 21:57:44,648 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-14 21:57:44,648 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1007385017:90 , srv1052944281:90 , srv1057833127:90 , srv112344151:90 , srv1168417272:90 , srv1323693174:90 , srv1347849854:90 , srv1571493607:90 , srv1639098071:90 , srv1761300056:90 , srv1769640147:90 , srv181474240:90 , srv182725658:90 , srv1848088716:90 , srv1859236641:90 , srv1963374216:90 , srv1965018439:90 , srv2011371813:90 , srv2014094588:90 , srv2044975645:90 , srv248651286:90 , srv301558743:90 , srv34170254:90 , srv381372942:90 , srv41970827:90 , srv560369459:90 , srv563257499:90 , srv590119527:90 , srv648293983:90 , srv982659110:90 }
2023-04-14 21:57:44,661 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=337 (was 478), ProcessCount=169 (was 169), AvailableMemoryMB=6559 (was 6606)