2023-03-07 09:54:13,734 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-03-07 09:54:14,042 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-03-07 09:54:14,148 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-03-07 09:54:14,203 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-07 09:54:14,210 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-03-07 09:54:14,223 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=257, ProcessCount=170, AvailableMemoryMB=10277
2023-03-07 09:54:14,225 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-07 09:54:14,225 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-03-07 09:54:14,354 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv101080358:84 , srv1011380957:84 , srv1028946699:84 , srv1063535394:84 , srv107298481:84 , srv1092425718:84 , srv1122085540:84 , srv1152810634:84 , srv1156775012:84 , srv1157673169:84 , srv1158338237:84 , srv117239829:84 , srv1173381161:84 , srv1222196536:84 , srv1247575489:84 , srv1265148488:84 , srv1293078187:84 , srv1334679812:84 , srv1385700352:84 , srv1397390715:84 , srv1403532217:84 , srv1405996783:84 , srv1446985755:84 , srv1491766564:84 , srv1506998038:84 , srv152134558:84 , srv1551395284:84 , srv1552715476:84 , srv1600059090:84 , srv1610456464:84 , srv1624485211:84 , srv162506397:84 , srv162588376:84 , srv1627113655:84 , srv1638329927:84 , srv1644207286:84 , srv1646692272:84 , srv1651276645:84 , srv1656246706:84 , srv1666937380:84 , srv1747921073:84 , srv1757873256:84 , srv1781691526:84 , srv1798353217:84 , srv1842160247:84 , srv1928027843:84 , srv193417578:84 , srv1935422385:84 , srv1962091544:84 , srv1968213966:84 , srv1998740276:84 , srv2003948294:84 , srv2028241358:84 , srv2053757270:84 , srv2079051674:84 , srv2087736366:84 , srv2105887136:84 , srv2110559134:84 , srv258858486:84 , srv260897827:84 , srv268437767:84 , srv293523758:84 , srv314164920:84 , srv330462821:84 , srv346188514:84 , srv356470940:84 , srv388542260:84 , srv40541717:84 , srv424375451:84 , srv426573852:84 , srv430711013:84 , srv4332958:84 , srv447581343:84 , srv486312449:84 , srv490695574:84 , srv539042663:84 , srv5712295:84 , srv596232304:84 , srv655966589:84 , srv661373138:84 , srv66284440:84 , srv707420397:84 , srv738441883:84 , srv770336084:84 , srv78677427:84 , srv822341059:84 , srv822901606:84 , srv835884042:84 , srv84023114:84 , srv84759593:84 , srv848422077:84 , srv866224804:84 , srv879642961:84 , srv917635937:84 , srv925178072:84 , srv938235024:84 , srv950291005:84 , srv970899856:84 , srv975970784:84 , srv1004681292:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-03-07 09:54:14,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-07 09:54:14,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1747921073=41, srv925178072=95, srv2079051674=55, srv430711013=71, srv1552715476=28, srv1998740276=51, srv1173381161=13, srv2105887136=57, srv1962091544=49, srv388542260=67, srv4332958=72, srv1666937380=40, srv40541717=68, srv866224804=92, srv822341059=86, srv1397390715=20, srv1122085540=7, srv1646692272=37, srv84023114=89, srv1551395284=27, srv1028946699=3, srv1334679812=18, srv1644207286=36, srv117239829=12, srv2087736366=56, srv5712295=77, srv426573852=70, srv835884042=88, srv486312449=74, srv107298481=5, srv152134558=26, srv1656246706=39, srv917635937=94, srv1385700352=19, srv770336084=84, srv1011380957=2, srv356470940=66, srv66284440=81, srv1935422385=48, srv1600059090=29, srv1405996783=22, srv1092425718=6, srv1004681292=0, srv2003948294=52, srv314164920=63, srv822901606=87, srv938235024=96, srv1158338237=11, srv1247575489=15, srv1928027843=46, srv2053757270=54, srv447581343=73, srv1781691526=43, srv1624485211=31, srv258858486=59, srv346188514=65, srv1968213966=50, srv268437767=61, srv490695574=75, srv1156775012=9, srv970899856=98, srv1446985755=23, srv539042663=76, srv162506397=32, srv707420397=82, srv330462821=64, srv879642961=93, srv1157673169=10, srv1222196536=14, srv1506998038=25, srv950291005=97, srv162588376=33, srv738441883=83, srv1638329927=35, srv101080358=1, srv78677427=85, srv975970784=99, srv1842160247=45, srv1798353217=44, srv1063535394=4, srv193417578=47, srv655966589=79, srv596232304=78, srv1757873256=42, srv1610456464=30, srv2110559134=58, srv661373138=80, srv848422077=91, srv1403532217=21, srv84759593=90, srv1293078187=17, srv1152810634=8, srv2028241358=53, srv260897827=60, srv424375451=69, srv1491766564=24, srv1265148488=16, srv1651276645=38, srv1627113655=34, srv293523758=62} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-07 09:54:14,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-07 09:54:14,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-07 09:54:14,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-07 09:54:14,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-07 09:54:14,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-07 09:54:14,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-07 09:54:14,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-07 09:54:14,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-07 09:54:14,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-07 09:54:14,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-07 09:54:14,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-07 09:54:14,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-07 09:54:14,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-07 09:54:14,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-07 09:54:14,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-07 09:54:14,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-07 09:54:14,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-07 09:54:14,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-07 09:54:14,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-07 09:54:14,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-07 09:54:14,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-07 09:54:14,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-07 09:54:14,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-07 09:54:14,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-07 09:54:14,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-07 09:54:14,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-07 09:54:14,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-07 09:54:14,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-07 09:54:14,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-07 09:54:14,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-07 09:54:14,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-07 09:54:14,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-07 09:54:14,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-07 09:54:14,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-07 09:54:14,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-07 09:54:14,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-07 09:54:14,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-07 09:54:14,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-07 09:54:14,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-07 09:54:14,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-07 09:54:14,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-07 09:54:14,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-07 09:54:14,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-07 09:54:14,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-07 09:54:14,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-07 09:54:14,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-07 09:54:14,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-07 09:54:14,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-07 09:54:14,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-07 09:54:14,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-07 09:54:14,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-07 09:54:14,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-07 09:54:14,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-07 09:54:14,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-07 09:54:14,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-07 09:54:14,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-07 09:54:14,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-07 09:54:14,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-07 09:54:14,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-07 09:54:14,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-07 09:54:14,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-07 09:54:14,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-07 09:54:14,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-07 09:54:14,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-07 09:54:14,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-07 09:54:14,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-07 09:54:14,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-07 09:54:14,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-07 09:54:14,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-07 09:54:14,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-07 09:54:14,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-07 09:54:14,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-07 09:54:14,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-07 09:54:14,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-07 09:54:14,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-07 09:54:14,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-07 09:54:14,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-07 09:54:14,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-07 09:54:14,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-07 09:54:14,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-07 09:54:14,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-07 09:54:14,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-07 09:54:14,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-07 09:54:14,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-07 09:54:14,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-07 09:54:14,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-07 09:54:14,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-07 09:54:14,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-07 09:54:14,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-07 09:54:14,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-07 09:54:14,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-07 09:54:14,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-07 09:54:14,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-07 09:54:14,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-07 09:54:14,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-07 09:54:14,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-07 09:54:14,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-07 09:54:14,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-07 09:54:14,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-07 09:54:14,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-07 09:54:14,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-07 09:54:14,628 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-07 09:54:14,629 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-07 09:54:14,629 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-07 09:54:14,904 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 416 ms to try 5000 different iterations.  Found a solution that moves 4737 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1771052194935166. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0404029461131274, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05736644568830364, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5263333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0404029461131274, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17191666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24266666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-07 09:54:14,937 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv107298481:82 , srv1405996783:82 , srv848422077:82 , srv1247575489:83 , srv162506397:83 , srv1656246706:83 , srv1998740276:83 , srv2053757270:83 , srv2079051674:83 , srv2087736366:83 , srv2105887136:83 , srv2110559134:83 , srv258858486:83 , srv426573852:83 , srv84023114:83 , srv84759593:83 , srv950291005:83 , srv1011380957:84 , srv1152810634:84 , srv1157673169:84 , srv117239829:84 , srv1397390715:84 , srv1551395284:84 , srv1781691526:84 , srv293523758:84 , srv430711013:84 , srv486312449:84 , srv661373138:84 , srv822341059:84 , srv1063535394:85 , srv1334679812:85 , srv1552715476:85 , srv162588376:85 , srv1747921073:85 , srv1962091544:85 , srv1968213966:85 , srv2028241358:85 , srv388542260:85 , srv40541717:85 , srv490695574:85 , srv5712295:85 , srv655966589:85 , srv707420397:85 , srv970899856:85 , srv1156775012:86 , srv1293078187:86 , srv1446985755:86 , srv1506998038:86 , srv1627113655:86 , srv1798353217:86 , srv193417578:86 , srv2003948294:86 , srv260897827:86 , srv879642961:86 , srv101080358:87 , srv1222196536:87 , srv1600059090:87 , srv1842160247:87 , srv596232304:87 , srv66284440:87 , srv738441883:87 , srv770336084:87 , srv78677427:87 , srv822901606:87 , srv1122085540:88 , srv1403532217:88 , srv1610456464:88 , srv1624485211:88 , srv1646692272:88 , srv1928027843:88 , srv330462821:88 , srv346188514:88 , srv539042663:88 , srv938235024:88 , srv1158338237:89 , srv1173381161:89 , srv1265148488:89 , srv1666937380:89 , srv1757873256:89 , srv268437767:89 , srv4332958:89 , srv447581343:89 , srv835884042:89 , srv925178072:89 , srv975970784:89 , srv1092425718:90 , srv1385700352:90 , srv152134558:90 , srv1638329927:90 , srv1651276645:90 , srv1935422385:90 , srv424375451:90 , srv866224804:90 , srv917635937:90 , srv1491766564:91 , srv1644207286:91 , srv1028946699:92 , srv314164920:92 , srv356470940:95 , srv1004681292:449 }
2023-03-07 09:54:14,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-07 09:54:14,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1747921073=41, srv925178072=95, srv2079051674=55, srv430711013=71, srv1552715476=28, srv1998740276=51, srv1173381161=13, srv2105887136=57, srv1962091544=49, srv388542260=67, srv4332958=72, srv1666937380=40, srv40541717=68, srv866224804=92, srv822341059=86, srv1397390715=20, srv1122085540=7, srv1646692272=37, srv84023114=89, srv1551395284=27, srv1028946699=3, srv1334679812=18, srv1644207286=36, srv117239829=12, srv2087736366=56, srv5712295=77, srv426573852=70, srv835884042=88, srv486312449=74, srv107298481=5, srv152134558=26, srv1656246706=39, srv917635937=94, srv1385700352=19, srv770336084=84, srv1011380957=2, srv356470940=66, srv66284440=81, srv1935422385=48, srv1600059090=29, srv1405996783=22, srv1092425718=6, srv1004681292=0, srv2003948294=52, srv314164920=63, srv822901606=87, srv938235024=96, srv1158338237=11, srv1247575489=15, srv1928027843=46, srv2053757270=54, srv447581343=73, srv1781691526=43, srv1624485211=31, srv258858486=59, srv346188514=65, srv1968213966=50, srv268437767=61, srv490695574=75, srv1156775012=9, srv970899856=98, srv1446985755=23, srv539042663=76, srv162506397=32, srv707420397=82, srv330462821=64, srv879642961=93, srv1157673169=10, srv1222196536=14, srv1506998038=25, srv950291005=97, srv162588376=33, srv738441883=83, srv1638329927=35, srv101080358=1, srv78677427=85, srv975970784=99, srv1842160247=45, srv1798353217=44, srv1063535394=4, srv193417578=47, srv655966589=79, srv596232304=78, srv1757873256=42, srv1610456464=30, srv2110559134=58, srv661373138=80, srv848422077=91, srv1403532217=21, srv84759593=90, srv1293078187=17, srv1152810634=8, srv2028241358=53, srv260897827=60, srv424375451=69, srv1491766564=24, srv1265148488=16, srv1651276645=38, srv1627113655=34, srv293523758=62} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-07 09:54:14,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-07 09:54:14,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-07 09:54:14,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-07 09:54:14,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-07 09:54:14,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-07 09:54:14,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-07 09:54:14,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-07 09:54:14,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-07 09:54:14,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-07 09:54:14,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-07 09:54:14,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-07 09:54:14,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-07 09:54:14,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-07 09:54:14,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-07 09:54:14,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-07 09:54:14,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-07 09:54:14,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-07 09:54:14,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-07 09:54:14,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-07 09:54:14,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-07 09:54:14,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-07 09:54:14,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-07 09:54:14,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-07 09:54:14,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-07 09:54:14,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-07 09:54:14,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-07 09:54:14,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-07 09:54:14,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-07 09:54:14,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-07 09:54:14,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-07 09:54:14,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-07 09:54:14,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-07 09:54:14,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-07 09:54:14,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-07 09:54:14,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-07 09:54:14,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-07 09:54:14,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-07 09:54:14,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-07 09:54:14,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-07 09:54:14,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-07 09:54:14,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-07 09:54:14,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-07 09:54:14,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-07 09:54:14,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-07 09:54:14,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-07 09:54:14,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-07 09:54:14,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-07 09:54:14,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-07 09:54:14,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-07 09:54:14,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-07 09:54:14,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-07 09:54:14,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-07 09:54:14,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-07 09:54:14,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-07 09:54:14,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-07 09:54:14,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-07 09:54:14,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-07 09:54:14,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-07 09:54:14,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-07 09:54:14,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-07 09:54:14,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-07 09:54:14,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-07 09:54:14,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-07 09:54:14,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-07 09:54:14,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-07 09:54:14,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-07 09:54:14,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-07 09:54:14,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-07 09:54:14,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-07 09:54:14,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-07 09:54:14,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-07 09:54:14,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-07 09:54:14,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-07 09:54:14,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-07 09:54:14,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-07 09:54:14,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-07 09:54:14,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-07 09:54:14,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-07 09:54:14,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-07 09:54:14,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-07 09:54:14,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-07 09:54:14,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-07 09:54:14,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-07 09:54:14,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-07 09:54:14,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-07 09:54:14,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-07 09:54:14,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-07 09:54:14,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-07 09:54:14,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-07 09:54:14,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-07 09:54:14,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-07 09:54:14,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-07 09:54:14,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-07 09:54:14,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-07 09:54:14,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-07 09:54:14,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-07 09:54:14,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-07 09:54:14,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-07 09:54:14,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-07 09:54:14,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-07 09:54:14,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-07 09:54:15,157 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0404029461131274, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05736644568830364, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0404029461131274, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17191666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24266666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-07 09:54:15,158 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-07 09:54:15,158 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17707205031057746, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0404029461131274, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05736644568830364, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0404029461131274, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17191666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24266666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-07 09:54:15,418 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 412 ms to try 5000 different iterations.  Found a solution that moves 2600 regions; Going from a computed imbalance of 0.17707205031057746 to a new imbalance of 0.0135726967665663. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021830440917265614); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.029492527920518045, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.28888888888888886, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021830440917265614); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010583333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.042083333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-07 09:54:15,429 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv356470940:77 , srv1644207286:79 , srv314164920:79 , srv1491766564:80 , srv866224804:80 , srv1158338237:81 , srv1651276645:81 , srv1092425718:82 , srv1122085540:82 , srv152134558:82 , srv1610456464:82 , srv424375451:82 , srv4332958:82 , srv835884042:82 , srv1028946699:83 , srv1173381161:83 , srv1222196536:83 , srv1403532217:83 , srv1757873256:83 , srv268437767:83 , srv447581343:83 , srv917635937:83 , srv101080358:84 , srv1265148488:84 , srv1385700352:84 , srv1600059090:84 , srv1624485211:84 , srv1638329927:84 , srv1842160247:84 , srv1928027843:84 , srv1935422385:84 , srv78677427:84 , srv822901606:84 , srv925178072:84 , srv975970784:84 , srv1446985755:85 , srv1646692272:85 , srv1798353217:85 , srv2003948294:85 , srv346188514:85 , srv40541717:85 , srv490695574:85 , srv539042663:85 , srv770336084:85 , srv879642961:85 , srv1011380957:86 , srv1152810634:86 , srv1156775012:86 , srv1157673169:86 , srv1293078187:86 , srv1506998038:86 , srv1627113655:86 , srv193417578:86 , srv1968213966:86 , srv330462821:86 , srv486312449:86 , srv66284440:86 , srv738441883:86 , srv970899856:86 , srv1063535394:87 , srv117239829:87 , srv1397390715:87 , srv1747921073:87 , srv2028241358:87 , srv260897827:87 , srv388542260:87 , srv596232304:87 , srv661373138:87 , srv822341059:87 , srv938235024:87 , srv107298481:88 , srv1334679812:88 , srv1551395284:88 , srv1552715476:88 , srv162506397:88 , srv162588376:88 , srv1781691526:88 , srv293523758:88 , srv430711013:88 , srv655966589:88 , srv84023114:88 , srv1666937380:89 , srv1998740276:89 , srv2079051674:89 , srv2105887136:89 , srv2110559134:89 , srv5712295:89 , srv707420397:89 , srv84759593:89 , srv950291005:89 , srv1247575489:90 , srv1405996783:90 , srv1962091544:90 , srv2053757270:90 , srv258858486:90 , srv848422077:90 , srv2087736366:91 , srv426573852:91 , srv1656246706:92 , srv1004681292:519 }
2023-03-07 09:54:15,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-07 09:54:15,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1747921073=41, srv925178072=95, srv2079051674=55, srv430711013=71, srv1552715476=28, srv1998740276=51, srv1173381161=13, srv2105887136=57, srv1962091544=49, srv388542260=67, srv4332958=72, srv1666937380=40, srv40541717=68, srv866224804=92, srv822341059=86, srv1397390715=20, srv1122085540=7, srv1646692272=37, srv84023114=89, srv1551395284=27, srv1028946699=3, srv1334679812=18, srv1644207286=36, srv117239829=12, srv2087736366=56, srv5712295=77, srv426573852=70, srv835884042=88, srv486312449=74, srv107298481=5, srv152134558=26, srv1656246706=39, srv917635937=94, srv1385700352=19, srv770336084=84, srv1011380957=2, srv356470940=66, srv66284440=81, srv1935422385=48, srv1600059090=29, srv1405996783=22, srv1092425718=6, srv1004681292=0, srv2003948294=52, srv314164920=63, srv822901606=87, srv938235024=96, srv1158338237=11, srv1247575489=15, srv1928027843=46, srv2053757270=54, srv447581343=73, srv1781691526=43, srv1624485211=31, srv258858486=59, srv346188514=65, srv1968213966=50, srv268437767=61, srv490695574=75, srv1156775012=9, srv970899856=98, srv1446985755=23, srv539042663=76, srv162506397=32, srv707420397=82, srv330462821=64, srv879642961=93, srv1157673169=10, srv1222196536=14, srv1506998038=25, srv950291005=97, srv162588376=33, srv738441883=83, srv1638329927=35, srv101080358=1, srv78677427=85, srv975970784=99, srv1842160247=45, srv1798353217=44, srv1063535394=4, srv193417578=47, srv655966589=79, srv596232304=78, srv1757873256=42, srv1610456464=30, srv2110559134=58, srv661373138=80, srv848422077=91, srv1403532217=21, srv84759593=90, srv1293078187=17, srv1152810634=8, srv2028241358=53, srv260897827=60, srv424375451=69, srv1491766564=24, srv1265148488=16, srv1651276645=38, srv1627113655=34, srv293523758=62} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-07 09:54:15,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-07 09:54:15,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-07 09:54:15,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-07 09:54:15,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-07 09:54:15,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-07 09:54:15,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-07 09:54:15,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-07 09:54:15,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-07 09:54:15,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-07 09:54:15,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-07 09:54:15,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-07 09:54:15,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-07 09:54:15,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-07 09:54:15,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-07 09:54:15,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-07 09:54:15,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-07 09:54:15,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-07 09:54:15,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-07 09:54:15,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-07 09:54:15,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-07 09:54:15,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-07 09:54:15,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-07 09:54:15,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-07 09:54:15,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-07 09:54:15,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-07 09:54:15,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-07 09:54:15,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-07 09:54:15,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-07 09:54:15,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-07 09:54:15,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-07 09:54:15,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-07 09:54:15,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-07 09:54:15,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-07 09:54:15,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-07 09:54:15,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-07 09:54:15,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-07 09:54:15,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-07 09:54:15,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-07 09:54:15,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-07 09:54:15,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-07 09:54:15,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-07 09:54:15,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-07 09:54:15,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-07 09:54:15,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-07 09:54:15,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-07 09:54:15,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-07 09:54:15,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-07 09:54:15,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-07 09:54:15,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-07 09:54:15,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-07 09:54:15,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-07 09:54:15,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-07 09:54:15,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-07 09:54:15,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-07 09:54:15,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-07 09:54:15,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-07 09:54:15,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-07 09:54:15,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-07 09:54:15,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-07 09:54:15,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-07 09:54:15,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-07 09:54:15,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-07 09:54:15,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-07 09:54:15,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-07 09:54:15,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-07 09:54:15,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-07 09:54:15,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-07 09:54:15,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-07 09:54:15,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-07 09:54:15,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-07 09:54:15,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-07 09:54:15,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-07 09:54:15,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-07 09:54:15,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-07 09:54:15,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-07 09:54:15,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-07 09:54:15,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-07 09:54:15,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-07 09:54:15,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-07 09:54:15,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-07 09:54:15,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-07 09:54:15,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-07 09:54:15,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-07 09:54:15,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-07 09:54:15,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-07 09:54:15,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-07 09:54:15,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-07 09:54:15,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-07 09:54:15,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-07 09:54:15,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-07 09:54:15,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-07 09:54:15,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-07 09:54:15,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-07 09:54:15,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-07 09:54:15,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-07 09:54:15,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-07 09:54:15,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-07 09:54:15,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-07 09:54:15,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-07 09:54:15,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-07 09:54:15,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-07 09:54:15,588 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021830440917265614); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.029492527920518045, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021830440917265614); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010583333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.042083333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-07 09:54:15,589 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-07 09:54:15,589 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.01355449117745044, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021830440917265614); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.029492527920518045, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021830440917265614); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010583333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.042083333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-07 09:54:15,827 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 341 ms to try 5000 different iterations.  Found a solution that moves 976 regions; Going from a computed imbalance of 0.01355449117745044 to a new imbalance of 5.788112553582089E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0034346735168502606); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005058579404262188); MoveCostFunction : (multiplier=7.0, imbalance=0.10844444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0034346735168502606); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-07 09:54:15,829 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1666937380:79 , srv1028946699:82 , srv938235024:82 , srv1385700352:83 , srv1656246706:83 , srv1935422385:83 , srv330462821:83 , srv1092425718:84 , srv1265148488:84 , srv1638329927:84 , srv1962091544:84 , srv2087736366:84 , srv260897827:84 , srv426573852:84 , srv539042663:84 , srv5712295:84 , srv596232304:84 , srv707420397:84 , srv738441883:84 , srv1063535394:85 , srv1247575489:85 , srv1334679812:85 , srv1506998038:85 , srv1552715476:85 , srv162588376:85 , srv1646692272:85 , srv2053757270:85 , srv2079051674:85 , srv258858486:85 , srv346188514:85 , srv424375451:85 , srv447581343:85 , srv655966589:85 , srv848422077:85 , srv917635937:85 , srv925178072:85 , srv975970784:85 , srv1152810634:86 , srv1156775012:86 , srv1173381161:86 , srv1397390715:86 , srv1405996783:86 , srv1446985755:86 , srv152134558:86 , srv1600059090:86 , srv1624485211:86 , srv162506397:86 , srv1627113655:86 , srv1747921073:86 , srv1757873256:86 , srv1781691526:86 , srv1842160247:86 , srv1928027843:86 , srv193417578:86 , srv1998740276:86 , srv2003948294:86 , srv2028241358:86 , srv2105887136:86 , srv268437767:86 , srv293523758:86 , srv314164920:86 , srv356470940:86 , srv430711013:86 , srv661373138:86 , srv66284440:86 , srv770336084:86 , srv822341059:86 , srv835884042:86 , srv84759593:86 , srv950291005:86 , srv101080358:87 , srv1011380957:87 , srv107298481:87 , srv117239829:87 , srv1222196536:87 , srv1293078187:87 , srv1403532217:87 , srv1491766564:87 , srv1551395284:87 , srv1644207286:87 , srv1651276645:87 , srv1798353217:87 , srv1968213966:87 , srv2110559134:87 , srv388542260:87 , srv40541717:87 , srv4332958:87 , srv78677427:87 , srv822901606:87 , srv84023114:87 , srv866224804:87 , srv879642961:87 , srv970899856:87 , srv1122085540:88 , srv1157673169:88 , srv1158338237:88 , srv1610456464:88 , srv486312449:88 , srv490695574:88 , srv1004681292:520 }
2023-03-07 09:54:15,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-07 09:54:15,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1747921073=41, srv925178072=95, srv2079051674=55, srv430711013=71, srv1552715476=28, srv1998740276=51, srv1173381161=13, srv2105887136=57, srv1962091544=49, srv388542260=67, srv4332958=72, srv1666937380=40, srv40541717=68, srv866224804=92, srv822341059=86, srv1397390715=20, srv1122085540=7, srv1646692272=37, srv84023114=89, srv1551395284=27, srv1028946699=3, srv1334679812=18, srv1644207286=36, srv117239829=12, srv2087736366=56, srv5712295=77, srv426573852=70, srv835884042=88, srv486312449=74, srv107298481=5, srv152134558=26, srv1656246706=39, srv917635937=94, srv1385700352=19, srv770336084=84, srv1011380957=2, srv356470940=66, srv66284440=81, srv1935422385=48, srv1600059090=29, srv1405996783=22, srv1092425718=6, srv1004681292=0, srv2003948294=52, srv314164920=63, srv822901606=87, srv938235024=96, srv1158338237=11, srv1247575489=15, srv1928027843=46, srv2053757270=54, srv447581343=73, srv1781691526=43, srv1624485211=31, srv258858486=59, srv346188514=65, srv1968213966=50, srv268437767=61, srv490695574=75, srv1156775012=9, srv970899856=98, srv1446985755=23, srv539042663=76, srv162506397=32, srv707420397=82, srv330462821=64, srv879642961=93, srv1157673169=10, srv1222196536=14, srv1506998038=25, srv950291005=97, srv162588376=33, srv738441883=83, srv1638329927=35, srv101080358=1, srv78677427=85, srv975970784=99, srv1842160247=45, srv1798353217=44, srv1063535394=4, srv193417578=47, srv655966589=79, srv596232304=78, srv1757873256=42, srv1610456464=30, srv2110559134=58, srv661373138=80, srv848422077=91, srv1403532217=21, srv84759593=90, srv1293078187=17, srv1152810634=8, srv2028241358=53, srv260897827=60, srv424375451=69, srv1491766564=24, srv1265148488=16, srv1651276645=38, srv1627113655=34, srv293523758=62} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-07 09:54:15,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-07 09:54:15,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-07 09:54:15,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-07 09:54:15,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-07 09:54:15,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-07 09:54:15,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-07 09:54:15,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-07 09:54:15,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-07 09:54:15,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-07 09:54:15,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-07 09:54:15,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-07 09:54:15,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-07 09:54:15,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-07 09:54:15,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-07 09:54:15,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-07 09:54:15,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-07 09:54:15,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-07 09:54:15,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-07 09:54:15,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-07 09:54:15,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-07 09:54:15,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-07 09:54:15,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-07 09:54:15,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-07 09:54:15,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-07 09:54:15,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-07 09:54:15,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-07 09:54:15,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-07 09:54:15,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-07 09:54:15,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-07 09:54:15,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-07 09:54:15,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-07 09:54:15,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-07 09:54:15,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-07 09:54:15,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-07 09:54:15,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-07 09:54:15,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-07 09:54:15,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-07 09:54:15,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-07 09:54:15,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-07 09:54:15,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-07 09:54:15,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-07 09:54:15,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-07 09:54:15,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-07 09:54:15,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-07 09:54:15,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-07 09:54:15,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-07 09:54:15,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-07 09:54:15,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-07 09:54:15,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-07 09:54:15,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-07 09:54:15,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-07 09:54:15,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-07 09:54:15,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-07 09:54:15,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-07 09:54:15,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-07 09:54:15,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-07 09:54:15,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-07 09:54:15,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-07 09:54:15,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-07 09:54:15,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-07 09:54:15,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-07 09:54:15,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-07 09:54:15,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-07 09:54:15,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-07 09:54:15,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-07 09:54:15,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-07 09:54:15,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-07 09:54:15,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-07 09:54:15,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-07 09:54:15,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-07 09:54:15,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-07 09:54:15,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-07 09:54:15,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-07 09:54:15,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-07 09:54:15,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-07 09:54:15,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-07 09:54:15,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-07 09:54:15,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-07 09:54:15,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-07 09:54:15,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-07 09:54:15,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-07 09:54:15,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-07 09:54:15,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-07 09:54:15,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-07 09:54:15,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-07 09:54:15,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-07 09:54:15,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-07 09:54:15,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-07 09:54:15,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-07 09:54:15,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-07 09:54:15,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-07 09:54:15,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-07 09:54:15,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-07 09:54:15,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-07 09:54:15,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-07 09:54:15,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-07 09:54:15,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-07 09:54:15,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-07 09:54:15,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-07 09:54:15,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-07 09:54:15,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-07 09:54:15,956 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0034346735168502606); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005058579404262188); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0034346735168502606); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-07 09:54:15,956 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-07 09:54:15,956 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=5.719771572901019E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0034346735168502606); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005058579404262188); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0034346735168502606); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-07 09:54:16,291 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 433 ms to try 5000 different iterations.  Found a solution that moves 204 regions; Going from a computed imbalance of 5.719771572901019E-4 to a new imbalance of 1.2627998057391242E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.02266666666666667); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-07 09:54:16,292 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1158338237:83 , srv117239829:83 , srv1293078187:83 , srv1551395284:83 , srv1798353217:83 , srv1962091544:83 , srv388542260:83 , srv66284440:83 , srv101080358:84 , srv1028946699:84 , srv1122085540:84 , srv1156775012:84 , srv1247575489:84 , srv1265148488:84 , srv1334679812:84 , srv1403532217:84 , srv1491766564:84 , srv152134558:84 , srv1552715476:84 , srv1610456464:84 , srv1624485211:84 , srv1627113655:84 , srv1638329927:84 , srv1646692272:84 , srv1651276645:84 , srv1656246706:84 , srv1747921073:84 , srv1928027843:84 , srv193417578:84 , srv1968213966:84 , srv1998740276:84 , srv2028241358:84 , srv2053757270:84 , srv2087736366:84 , srv2105887136:84 , srv2110559134:84 , srv258858486:84 , srv293523758:84 , srv330462821:84 , srv346188514:84 , srv356470940:84 , srv426573852:84 , srv430711013:84 , srv4332958:84 , srv486312449:84 , srv490695574:84 , srv539042663:84 , srv5712295:84 , srv596232304:84 , srv655966589:84 , srv707420397:84 , srv770336084:84 , srv78677427:84 , srv822901606:84 , srv84023114:84 , srv84759593:84 , srv879642961:84 , srv917635937:84 , srv925178072:84 , srv950291005:84 , srv970899856:84 , srv975970784:84 , srv1011380957:85 , srv1063535394:85 , srv107298481:85 , srv1152810634:85 , srv1157673169:85 , srv1173381161:85 , srv1222196536:85 , srv1385700352:85 , srv1397390715:85 , srv1405996783:85 , srv1446985755:85 , srv1506998038:85 , srv1600059090:85 , srv162506397:85 , srv162588376:85 , srv1644207286:85 , srv1666937380:85 , srv1757873256:85 , srv1781691526:85 , srv1842160247:85 , srv1935422385:85 , srv2003948294:85 , srv2079051674:85 , srv260897827:85 , srv268437767:85 , srv314164920:85 , srv40541717:85 , srv424375451:85 , srv447581343:85 , srv661373138:85 , srv738441883:85 , srv822341059:85 , srv835884042:85 , srv848422077:85 , srv866224804:85 , srv938235024:85 , srv1092425718:86 , srv1004681292:654 }
2023-03-07 09:54:16,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-07 09:54:16,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1747921073=41, srv925178072=95, srv2079051674=55, srv430711013=71, srv1552715476=28, srv1998740276=51, srv1173381161=13, srv2105887136=57, srv1962091544=49, srv388542260=67, srv4332958=72, srv1666937380=40, srv40541717=68, srv866224804=92, srv822341059=86, srv1397390715=20, srv1122085540=7, srv1646692272=37, srv84023114=89, srv1551395284=27, srv1028946699=3, srv1334679812=18, srv1644207286=36, srv117239829=12, srv2087736366=56, srv5712295=77, srv426573852=70, srv835884042=88, srv486312449=74, srv107298481=5, srv152134558=26, srv1656246706=39, srv917635937=94, srv1385700352=19, srv770336084=84, srv1011380957=2, srv356470940=66, srv66284440=81, srv1935422385=48, srv1600059090=29, srv1405996783=22, srv1092425718=6, srv1004681292=0, srv2003948294=52, srv314164920=63, srv822901606=87, srv938235024=96, srv1158338237=11, srv1247575489=15, srv1928027843=46, srv2053757270=54, srv447581343=73, srv1781691526=43, srv1624485211=31, srv258858486=59, srv346188514=65, srv1968213966=50, srv268437767=61, srv490695574=75, srv1156775012=9, srv970899856=98, srv1446985755=23, srv539042663=76, srv162506397=32, srv707420397=82, srv330462821=64, srv879642961=93, srv1157673169=10, srv1222196536=14, srv1506998038=25, srv950291005=97, srv162588376=33, srv738441883=83, srv1638329927=35, srv101080358=1, srv78677427=85, srv975970784=99, srv1842160247=45, srv1798353217=44, srv1063535394=4, srv193417578=47, srv655966589=79, srv596232304=78, srv1757873256=42, srv1610456464=30, srv2110559134=58, srv661373138=80, srv848422077=91, srv1403532217=21, srv84759593=90, srv1293078187=17, srv1152810634=8, srv2028241358=53, srv260897827=60, srv424375451=69, srv1491766564=24, srv1265148488=16, srv1651276645=38, srv1627113655=34, srv293523758=62} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-07 09:54:16,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-07 09:54:16,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-07 09:54:16,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-07 09:54:16,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-07 09:54:16,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-07 09:54:16,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-07 09:54:16,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-07 09:54:16,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-07 09:54:16,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-07 09:54:16,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-07 09:54:16,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-07 09:54:16,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-07 09:54:16,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-07 09:54:16,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-07 09:54:16,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-07 09:54:16,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-07 09:54:16,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-07 09:54:16,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-07 09:54:16,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-07 09:54:16,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-07 09:54:16,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-07 09:54:16,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-07 09:54:16,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-07 09:54:16,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-07 09:54:16,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-07 09:54:16,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-07 09:54:16,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-07 09:54:16,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-07 09:54:16,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-07 09:54:16,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-07 09:54:16,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-07 09:54:16,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-07 09:54:16,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-07 09:54:16,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-07 09:54:16,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-07 09:54:16,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-07 09:54:16,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-07 09:54:16,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-07 09:54:16,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-07 09:54:16,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-07 09:54:16,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-07 09:54:16,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-07 09:54:16,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-07 09:54:16,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-07 09:54:16,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-07 09:54:16,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-07 09:54:16,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-07 09:54:16,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-07 09:54:16,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-07 09:54:16,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-07 09:54:16,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-07 09:54:16,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-07 09:54:16,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-07 09:54:16,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-07 09:54:16,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-07 09:54:16,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-07 09:54:16,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-07 09:54:16,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-07 09:54:16,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-07 09:54:16,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-07 09:54:16,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-07 09:54:16,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-07 09:54:16,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-07 09:54:16,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-07 09:54:16,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-07 09:54:16,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-07 09:54:16,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-07 09:54:16,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-07 09:54:16,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-07 09:54:16,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-07 09:54:16,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-07 09:54:16,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-07 09:54:16,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-07 09:54:16,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-07 09:54:16,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-07 09:54:16,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-07 09:54:16,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-07 09:54:16,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-07 09:54:16,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-07 09:54:16,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-07 09:54:16,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-07 09:54:16,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-07 09:54:16,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-07 09:54:16,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-07 09:54:16,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-07 09:54:16,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-07 09:54:16,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-07 09:54:16,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-07 09:54:16,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-07 09:54:16,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-07 09:54:16,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-07 09:54:16,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-07 09:54:16,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-07 09:54:16,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-07 09:54:16,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-07 09:54:16,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-07 09:54:16,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-07 09:54:16,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-07 09:54:16,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-07 09:54:16,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-07 09:54:16,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-07 09:54:16,417 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.119955952676234E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-07 09:54:16,417 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1158338237:83 , srv117239829:83 , srv1293078187:83 , srv1551395284:83 , srv1798353217:83 , srv1962091544:83 , srv388542260:83 , srv66284440:83 , srv101080358:84 , srv1028946699:84 , srv1122085540:84 , srv1156775012:84 , srv1247575489:84 , srv1265148488:84 , srv1334679812:84 , srv1403532217:84 , srv1491766564:84 , srv152134558:84 , srv1552715476:84 , srv1610456464:84 , srv1624485211:84 , srv1627113655:84 , srv1638329927:84 , srv1646692272:84 , srv1651276645:84 , srv1656246706:84 , srv1747921073:84 , srv1928027843:84 , srv193417578:84 , srv1968213966:84 , srv1998740276:84 , srv2028241358:84 , srv2053757270:84 , srv2087736366:84 , srv2105887136:84 , srv2110559134:84 , srv258858486:84 , srv293523758:84 , srv330462821:84 , srv346188514:84 , srv356470940:84 , srv426573852:84 , srv430711013:84 , srv4332958:84 , srv486312449:84 , srv490695574:84 , srv539042663:84 , srv5712295:84 , srv596232304:84 , srv655966589:84 , srv707420397:84 , srv770336084:84 , srv78677427:84 , srv822901606:84 , srv84023114:84 , srv84759593:84 , srv879642961:84 , srv917635937:84 , srv925178072:84 , srv950291005:84 , srv970899856:84 , srv975970784:84 , srv1011380957:85 , srv1063535394:85 , srv107298481:85 , srv1152810634:85 , srv1157673169:85 , srv1173381161:85 , srv1222196536:85 , srv1385700352:85 , srv1397390715:85 , srv1405996783:85 , srv1446985755:85 , srv1506998038:85 , srv1600059090:85 , srv162506397:85 , srv162588376:85 , srv1644207286:85 , srv1666937380:85 , srv1757873256:85 , srv1781691526:85 , srv1842160247:85 , srv1935422385:85 , srv2003948294:85 , srv2079051674:85 , srv260897827:85 , srv268437767:85 , srv314164920:85 , srv40541717:85 , srv424375451:85 , srv447581343:85 , srv661373138:85 , srv738441883:85 , srv822341059:85 , srv835884042:85 , srv848422077:85 , srv866224804:85 , srv938235024:85 , srv1092425718:86 , srv1004681292:654 }
2023-03-07 09:54:16,442 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=269 (was 257) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=9794 (was 10277)
2023-03-07 09:54:16,452 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=269, ProcessCount=170, AvailableMemoryMB=9794
2023-03-07 09:54:16,453 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-07 09:54:16,453 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-03-07 09:54:16,463 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1010271393:84 , srv1020823117:84 , srv117376910:84 , srv1272742547:84 , srv1294118704:84 , srv1401813935:84 , srv1675173824:84 , srv1685600981:84 , srv1726535243:84 , srv1749796652:84 , srv1787501799:84 , srv1868605739:84 , srv1922676589:84 , srv2001858540:84 , srv2019936604:84 , srv2030282117:84 , srv2110667092:84 , srv2114855968:84 , srv218447064:84 , srv50856989:84 , srv584850049:84 , srv620526223:84 , srv646139997:84 , srv739951896:84 , srv75705567:84 , srv849022276:84 , srv863328286:84 , srv959399996:84 , srv973667145:84 , srv208116696:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-03-07 09:54:16,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-07 09:54:16,465 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1020823117=1, srv2001858540=13, srv646139997=23, srv863328286=27, srv1922676589=12, srv1294118704=4, srv1868605739=11, srv50856989=20, srv959399996=28, srv1010271393=0, srv75705567=25, srv1726535243=8, srv117376910=2, srv2110667092=17, srv1749796652=9, srv218447064=19, srv584850049=21, srv1675173824=6, srv1272742547=3, srv1401813935=5, srv620526223=22, srv2030282117=15, srv973667145=29, srv2019936604=14, srv1787501799=10, srv2114855968=18, srv1685600981=7, srv208116696=16, srv739951896=24, srv849022276=26} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-07 09:54:16,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-07 09:54:16,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-07 09:54:16,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-07 09:54:16,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-07 09:54:16,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-07 09:54:16,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-07 09:54:16,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-07 09:54:16,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-07 09:54:16,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-07 09:54:16,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-07 09:54:16,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-07 09:54:16,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-07 09:54:16,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-07 09:54:16,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-07 09:54:16,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-07 09:54:16,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-07 09:54:16,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-07 09:54:16,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-07 09:54:16,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-07 09:54:16,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-07 09:54:16,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-07 09:54:16,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-07 09:54:16,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-07 09:54:16,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-07 09:54:16,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-07 09:54:16,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-07 09:54:16,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-07 09:54:16,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-07 09:54:16,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-07 09:54:16,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-07 09:54:16,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-03-07 09:54:16,493 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0819711533018697, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-07 09:54:16,494 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912451331691731, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0819711533018697, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-03-07 09:56:16,480 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6072119 different iterations.  Found a solution that moves 1844 regions; Going from a computed imbalance of 0.9912451331691731 to a new imbalance of 4.303987990979898E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.682962962962963, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-07 09:56:16,486 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1010271393:90 , srv1020823117:90 , srv117376910:90 , srv1272742547:90 , srv1294118704:90 , srv1401813935:90 , srv1675173824:90 , srv1685600981:90 , srv1726535243:90 , srv1749796652:90 , srv1787501799:90 , srv1868605739:90 , srv1922676589:90 , srv2001858540:90 , srv2019936604:90 , srv2030282117:90 , srv208116696:90 , srv2110667092:90 , srv2114855968:90 , srv218447064:90 , srv50856989:90 , srv584850049:90 , srv620526223:90 , srv646139997:90 , srv739951896:90 , srv75705567:90 , srv849022276:90 , srv863328286:90 , srv959399996:90 , srv973667145:90 }
2023-03-07 09:56:16,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-07 09:56:16,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1020823117=1, srv2001858540=13, srv646139997=23, srv863328286=27, srv1922676589=12, srv1294118704=4, srv1868605739=11, srv50856989=20, srv959399996=28, srv1010271393=0, srv75705567=25, srv1726535243=8, srv117376910=2, srv2110667092=17, srv1749796652=9, srv218447064=19, srv584850049=21, srv1675173824=6, srv1272742547=3, srv1401813935=5, srv620526223=22, srv2030282117=15, srv973667145=29, srv2019936604=14, srv1787501799=10, srv2114855968=18, srv1685600981=7, srv208116696=16, srv739951896=24, srv849022276=26} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-07 09:56:16,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-07 09:56:16,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-07 09:56:16,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-07 09:56:16,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-07 09:56:16,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-07 09:56:16,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-07 09:56:16,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-07 09:56:16,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-07 09:56:16,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-07 09:56:16,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-07 09:56:16,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-07 09:56:16,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-07 09:56:16,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-07 09:56:16,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-07 09:56:16,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-07 09:56:16,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-07 09:56:16,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-07 09:56:16,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-07 09:56:16,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-07 09:56:16,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-07 09:56:16,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-07 09:56:16,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-07 09:56:16,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-07 09:56:16,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-07 09:56:16,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-07 09:56:16,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-07 09:56:16,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-07 09:56:16,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-07 09:56:16,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-07 09:56:16,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-07 09:56:16,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-03-07 09:56:16,505 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-07 09:56:16,505 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1010271393:90 , srv1020823117:90 , srv117376910:90 , srv1272742547:90 , srv1294118704:90 , srv1401813935:90 , srv1675173824:90 , srv1685600981:90 , srv1726535243:90 , srv1749796652:90 , srv1787501799:90 , srv1868605739:90 , srv1922676589:90 , srv2001858540:90 , srv2019936604:90 , srv2030282117:90 , srv208116696:90 , srv2110667092:90 , srv2114855968:90 , srv218447064:90 , srv50856989:90 , srv584850049:90 , srv620526223:90 , srv646139997:90 , srv739951896:90 , srv75705567:90 , srv849022276:90 , srv863328286:90 , srv959399996:90 , srv973667145:90 }
2023-03-07 09:56:16,519 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=163 (was 269), ProcessCount=170 (was 170), AvailableMemoryMB=9496 (was 9794)