2023-04-17 23:00:24,130 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-04-17 23:00:24,395 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-04-17 23:00:24,501 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-04-17 23:00:24,550 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-17 23:00:24,556 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-04-17 23:00:24,567 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=150, ProcessCount=167, AvailableMemoryMB=12398
2023-04-17 23:00:24,570 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-17 23:00:24,570 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-04-17 23:00:24,689 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1020252823:84 , srv1034632116:84 , srv1052008619:84 , srv1083502833:84 , srv1095996807:84 , srv1114514665:84 , srv1122203348:84 , srv1143933894:84 , srv115363311:84 , srv1220173609:84 , srv123933117:84 , srv1254916663:84 , srv1260954345:84 , srv1266535654:84 , srv134179578:84 , srv1356543144:84 , srv1374119815:84 , srv1378533930:84 , srv1394773989:84 , srv1441433420:84 , srv1482670322:84 , srv1519802789:84 , srv1544157149:84 , srv155071402:84 , srv1566678535:84 , srv1569304287:84 , srv1572302990:84 , srv1612804385:84 , srv1660488987:84 , srv1669570045:84 , srv1676230324:84 , srv1691677296:84 , srv1720967160:84 , srv1725937896:84 , srv1727896613:84 , srv1741829077:84 , srv1761901900:84 , srv1853777415:84 , srv1856110441:84 , srv1893173979:84 , srv1925290858:84 , srv1926685836:84 , srv1934188582:84 , srv1934474037:84 , srv1967577809:84 , srv1969307767:84 , srv1980540833:84 , srv2029297133:84 , srv2031148266:84 , srv2032044132:84 , srv2036089316:84 , srv2059596851:84 , srv2065623202:84 , srv2085120895:84 , srv2085563553:84 , srv2086302170:84 , srv2086875034:84 , srv2088752251:84 , srv2094387873:84 , srv2109044521:84 , srv211285569:84 , srv252515376:84 , srv267255830:84 , srv294802281:84 , srv297475273:84 , srv313685128:84 , srv315542634:84 , srv332210952:84 , srv382793774:84 , srv391851444:84 , srv399787272:84 , srv416194638:84 , srv451023979:84 , srv457194205:84 , srv462520556:84 , srv469253582:84 , srv502215884:84 , srv506279912:84 , srv519935400:84 , srv538287274:84 , srv608239356:84 , srv611080261:84 , srv61713732:84 , srv639580070:84 , srv66690928:84 , srv690157299:84 , srv708601223:84 , srv739408904:84 , srv772931035:84 , srv834792470:84 , srv856992338:84 , srv861360450:84 , srv904514984:84 , srv916486289:84 , srv962989538:84 , srv967507115:84 , srv967767579:84 , srv969346810:84 , srv984399391:84 , srv2058426743:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-04-17 23:00:24,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-17 23:00:24,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv861360450=92, srv399787272=71, srv1394773989=18, srv1925290858=40, srv1254916663=11, srv2086302170=56, srv519935400=79, srv313685128=66, srv1356543144=15, srv834792470=90, srv1741829077=35, srv1020252823=0, srv1856110441=38, srv2094387873=59, srv2031148266=48, srv123933117=10, srv1761901900=36, srv772931035=89, srv1727896613=34, srv2088752251=58, srv1926685836=41, srv294802281=64, srv1034632116=1, srv134179578=14, srv1260954345=12, srv984399391=99, srv1122203348=6, srv1669570045=29, srv1114514665=5, srv267255830=63, srv457194205=74, srv61713732=83, srv252515376=62, srv155071402=23, srv469253582=76, srv1934188582=42, srv1095996807=4, srv1676230324=30, srv1482670322=20, srv1612804385=27, srv2036089316=50, srv506279912=78, srv315542634=67, srv1969307767=45, srv451023979=73, srv1220173609=9, srv962989538=95, srv1052008619=2, srv967507115=96, srv967767579=97, srv1544157149=22, srv66690928=85, srv211285569=61, srv538287274=80, srv1720967160=32, srv2109044521=60, srv332210952=68, srv391851444=70, srv2032044132=49, srv2058426743=51, srv462520556=75, srv1378533930=17, srv1266535654=13, srv2059596851=52, srv1566678535=24, srv690157299=86, srv739408904=88, srv2085120895=54, srv916486289=94, srv1967577809=44, srv1569304287=25, srv608239356=81, srv969346810=98, srv1980540833=46, srv2029297133=47, srv297475273=65, srv611080261=82, srv639580070=84, srv1660488987=28, srv708601223=87, srv856992338=91, srv2085563553=55, srv1691677296=31, srv1374119815=16, srv1441433420=19, srv2086875034=57, srv1143933894=7, srv1893173979=39, srv1519802789=21, srv1083502833=3, srv1572302990=26, srv382793774=69, srv502215884=77, srv1934474037=43, srv1725937896=33, srv115363311=8, srv2065623202=53, srv904514984=93, srv416194638=72, srv1853777415=37} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-17 23:00:24,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-17 23:00:24,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-17 23:00:24,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-17 23:00:24,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-17 23:00:24,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-17 23:00:24,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-17 23:00:24,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-17 23:00:24,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-17 23:00:24,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-17 23:00:24,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-17 23:00:24,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-17 23:00:24,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-17 23:00:24,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-17 23:00:24,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-17 23:00:24,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-17 23:00:24,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-17 23:00:24,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-17 23:00:24,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-17 23:00:24,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-17 23:00:24,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-17 23:00:24,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-17 23:00:24,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-17 23:00:24,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-17 23:00:24,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-17 23:00:24,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-17 23:00:24,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-17 23:00:24,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-17 23:00:24,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-17 23:00:24,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-17 23:00:24,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-17 23:00:24,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-17 23:00:24,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-17 23:00:24,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-17 23:00:24,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-17 23:00:24,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-17 23:00:24,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-17 23:00:24,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-17 23:00:24,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-17 23:00:24,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-17 23:00:24,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-17 23:00:24,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-17 23:00:24,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-17 23:00:24,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-17 23:00:24,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-17 23:00:24,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-17 23:00:24,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-17 23:00:24,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-17 23:00:24,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-17 23:00:24,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-17 23:00:24,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-17 23:00:24,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-17 23:00:24,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-17 23:00:24,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-17 23:00:24,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-17 23:00:24,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-17 23:00:24,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-17 23:00:24,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-17 23:00:24,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-17 23:00:24,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-17 23:00:24,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-17 23:00:24,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-17 23:00:24,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-17 23:00:24,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-17 23:00:24,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-17 23:00:24,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-17 23:00:24,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-17 23:00:24,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-17 23:00:24,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-17 23:00:24,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-17 23:00:24,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-17 23:00:24,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-17 23:00:24,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-17 23:00:24,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-17 23:00:24,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-17 23:00:24,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-17 23:00:24,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-17 23:00:24,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-17 23:00:24,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-17 23:00:24,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-17 23:00:24,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-17 23:00:24,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-17 23:00:24,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-17 23:00:24,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-17 23:00:24,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-17 23:00:24,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-17 23:00:24,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-17 23:00:24,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-17 23:00:24,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-17 23:00:24,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-17 23:00:24,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-17 23:00:24,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-17 23:00:24,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-17 23:00:24,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-17 23:00:24,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-17 23:00:24,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-17 23:00:24,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-17 23:00:24,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-17 23:00:24,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-17 23:00:24,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-17 23:00:24,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-17 23:00:24,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-17 23:00:24,950 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-17 23:00:24,950 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-17 23:00:24,950 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-17 23:00:25,254 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 439 ms to try 5000 different iterations.  Found a solution that moves 4699 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1752827698344003. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040273413982424254, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.061033007941288554, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5221111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040273413982424254, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17033333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23808333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-17 23:00:25,271 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1519802789:82 , srv1853777415:82 , srv2029297133:82 , srv1083502833:83 , srv134179578:83 , srv1378533930:83 , srv967767579:83 , srv1020252823:84 , srv1052008619:84 , srv1220173609:84 , srv155071402:84 , srv1934474037:84 , srv2085120895:84 , srv252515376:84 , srv267255830:84 , srv294802281:84 , srv382793774:84 , srv416194638:84 , srv457194205:84 , srv502215884:84 , srv61713732:84 , srv739408904:84 , srv772931035:84 , srv904514984:84 , srv1034632116:85 , srv1122203348:85 , srv1143933894:85 , srv1356543144:85 , srv1572302990:85 , srv1660488987:85 , srv1761901900:85 , srv1856110441:85 , srv2031148266:85 , srv2036089316:85 , srv2086302170:85 , srv2086875034:85 , srv2088752251:85 , srv2109044521:85 , srv297475273:85 , srv469253582:85 , srv834792470:85 , srv861360450:85 , srv984399391:85 , srv123933117:86 , srv1260954345:86 , srv1394773989:86 , srv1441433420:86 , srv1566678535:86 , srv1569304287:86 , srv1720967160:86 , srv1725937896:86 , srv1727896613:86 , srv2065623202:86 , srv313685128:86 , srv519935400:86 , srv538287274:86 , srv611080261:86 , srv66690928:86 , srv967507115:86 , srv1254916663:87 , srv1374119815:87 , srv1482670322:87 , srv1612804385:87 , srv1691677296:87 , srv1934188582:87 , srv2032044132:87 , srv462520556:87 , srv608239356:87 , srv962989538:87 , srv1114514665:88 , srv115363311:88 , srv1544157149:88 , srv1669570045:88 , srv1969307767:88 , srv2059596851:88 , srv391851444:88 , srv451023979:88 , srv506279912:88 , srv639580070:88 , srv856992338:88 , srv916486289:88 , srv1095996807:89 , srv1926685836:89 , srv1980540833:89 , srv399787272:89 , srv1741829077:90 , srv1925290858:90 , srv2085563553:90 , srv315542634:90 , srv332210952:90 , srv969346810:90 , srv1266535654:91 , srv1893173979:91 , srv1967577809:91 , srv690157299:91 , srv708601223:91 , srv1676230324:92 , srv2094387873:93 , srv211285569:93 , srv2058426743:448 }
2023-04-17 23:00:25,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-17 23:00:25,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv861360450=92, srv399787272=71, srv1394773989=18, srv1925290858=40, srv1254916663=11, srv2086302170=56, srv519935400=79, srv313685128=66, srv1356543144=15, srv834792470=90, srv1741829077=35, srv1020252823=0, srv1856110441=38, srv2094387873=59, srv2031148266=48, srv123933117=10, srv1761901900=36, srv772931035=89, srv1727896613=34, srv2088752251=58, srv1926685836=41, srv294802281=64, srv1034632116=1, srv134179578=14, srv1260954345=12, srv984399391=99, srv1122203348=6, srv1669570045=29, srv1114514665=5, srv267255830=63, srv457194205=74, srv61713732=83, srv252515376=62, srv155071402=23, srv469253582=76, srv1934188582=42, srv1095996807=4, srv1676230324=30, srv1482670322=20, srv1612804385=27, srv2036089316=50, srv506279912=78, srv315542634=67, srv1969307767=45, srv451023979=73, srv1220173609=9, srv962989538=95, srv1052008619=2, srv967507115=96, srv967767579=97, srv1544157149=22, srv66690928=85, srv211285569=61, srv538287274=80, srv1720967160=32, srv2109044521=60, srv332210952=68, srv391851444=70, srv2032044132=49, srv2058426743=51, srv462520556=75, srv1378533930=17, srv1266535654=13, srv2059596851=52, srv1566678535=24, srv690157299=86, srv739408904=88, srv2085120895=54, srv916486289=94, srv1967577809=44, srv1569304287=25, srv608239356=81, srv969346810=98, srv1980540833=46, srv2029297133=47, srv297475273=65, srv611080261=82, srv639580070=84, srv1660488987=28, srv708601223=87, srv856992338=91, srv2085563553=55, srv1691677296=31, srv1374119815=16, srv1441433420=19, srv2086875034=57, srv1143933894=7, srv1893173979=39, srv1519802789=21, srv1083502833=3, srv1572302990=26, srv382793774=69, srv502215884=77, srv1934474037=43, srv1725937896=33, srv115363311=8, srv2065623202=53, srv904514984=93, srv416194638=72, srv1853777415=37} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-17 23:00:25,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-17 23:00:25,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-17 23:00:25,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-17 23:00:25,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-17 23:00:25,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-17 23:00:25,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-17 23:00:25,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-17 23:00:25,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-17 23:00:25,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-17 23:00:25,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-17 23:00:25,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-17 23:00:25,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-17 23:00:25,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-17 23:00:25,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-17 23:00:25,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-17 23:00:25,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-17 23:00:25,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-17 23:00:25,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-17 23:00:25,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-17 23:00:25,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-17 23:00:25,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-17 23:00:25,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-17 23:00:25,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-17 23:00:25,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-17 23:00:25,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-17 23:00:25,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-17 23:00:25,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-17 23:00:25,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-17 23:00:25,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-17 23:00:25,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-17 23:00:25,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-17 23:00:25,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-17 23:00:25,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-17 23:00:25,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-17 23:00:25,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-17 23:00:25,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-17 23:00:25,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-17 23:00:25,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-17 23:00:25,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-17 23:00:25,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-17 23:00:25,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-17 23:00:25,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-17 23:00:25,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-17 23:00:25,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-17 23:00:25,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-17 23:00:25,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-17 23:00:25,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-17 23:00:25,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-17 23:00:25,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-17 23:00:25,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-17 23:00:25,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-17 23:00:25,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-17 23:00:25,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-17 23:00:25,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-17 23:00:25,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-17 23:00:25,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-17 23:00:25,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-17 23:00:25,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-17 23:00:25,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-17 23:00:25,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-17 23:00:25,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-17 23:00:25,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-17 23:00:25,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-17 23:00:25,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-17 23:00:25,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-17 23:00:25,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-17 23:00:25,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-17 23:00:25,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-17 23:00:25,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-17 23:00:25,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-17 23:00:25,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-17 23:00:25,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-17 23:00:25,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-17 23:00:25,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-17 23:00:25,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-17 23:00:25,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-17 23:00:25,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-17 23:00:25,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-17 23:00:25,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-17 23:00:25,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-17 23:00:25,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-17 23:00:25,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-17 23:00:25,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-17 23:00:25,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-17 23:00:25,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-17 23:00:25,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-17 23:00:25,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-17 23:00:25,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-17 23:00:25,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-17 23:00:25,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-17 23:00:25,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-17 23:00:25,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-17 23:00:25,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-17 23:00:25,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-17 23:00:25,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-17 23:00:25,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-17 23:00:25,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-17 23:00:25,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-17 23:00:25,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-17 23:00:25,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-17 23:00:25,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-17 23:00:25,428 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040273413982424254, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.061033007941288554, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040273413982424254, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17033333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23808333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-17 23:00:25,429 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-17 23:00:25,429 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.1752498667331482, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040273413982424254, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.061033007941288554, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040273413982424254, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17033333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23808333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-17 23:00:25,625 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 309 ms to try 5000 different iterations.  Found a solution that moves 2650 regions; Going from a computed imbalance of 0.1752498667331482 to a new imbalance of 0.014983433292166072. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020597146021777486); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.034135137789004136, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29444444444444445, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020597146021777486); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043416666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-17 23:00:25,636 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1676230324:78 , srv211285569:79 , srv1967577809:80 , srv969346810:80 , srv115363311:81 , srv2085563553:81 , srv1612804385:82 , srv1893173979:82 , srv315542634:82 , srv332210952:82 , srv399787272:82 , srv690157299:82 , srv708601223:82 , srv1266535654:83 , srv1374119815:83 , srv1544157149:83 , srv1691677296:83 , srv1741829077:83 , srv1925290858:83 , srv1934188582:83 , srv2059596851:83 , srv2094387873:83 , srv451023979:83 , srv506279912:83 , srv916486289:83 , srv1095996807:84 , srv1727896613:84 , srv1969307767:84 , srv1980540833:84 , srv1114514665:85 , srv1254916663:85 , srv1260954345:85 , srv1569304287:85 , srv1856110441:85 , srv1926685836:85 , srv2032044132:85 , srv391851444:85 , srv519935400:85 , srv611080261:85 , srv639580070:85 , srv856992338:85 , srv962989538:85 , srv1143933894:86 , srv123933117:86 , srv1441433420:86 , srv1566678535:86 , srv1572302990:86 , srv1669570045:86 , srv1720967160:86 , srv2031148266:86 , srv2086875034:86 , srv416194638:86 , srv462520556:86 , srv61713732:86 , srv1034632116:87 , srv1052008619:87 , srv1394773989:87 , srv2085120895:87 , srv267255830:87 , srv294802281:87 , srv297475273:87 , srv469253582:87 , srv538287274:87 , srv66690928:87 , srv739408904:87 , srv834792470:87 , srv967507115:87 , srv984399391:87 , srv1122203348:88 , srv1220173609:88 , srv134179578:88 , srv1356543144:88 , srv1482670322:88 , srv155071402:88 , srv1725937896:88 , srv1761901900:88 , srv2036089316:88 , srv2065623202:88 , srv2086302170:88 , srv2109044521:88 , srv313685128:88 , srv382793774:88 , srv608239356:88 , srv772931035:88 , srv861360450:88 , srv904514984:88 , srv1020252823:89 , srv1660488987:89 , srv1934474037:89 , srv2029297133:89 , srv502215884:89 , srv967767579:89 , srv1519802789:90 , srv1853777415:90 , srv2088752251:90 , srv457194205:90 , srv1083502833:91 , srv1378533930:91 , srv252515376:91 , srv2058426743:509 }
2023-04-17 23:00:25,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-17 23:00:25,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv861360450=92, srv399787272=71, srv1394773989=18, srv1925290858=40, srv1254916663=11, srv2086302170=56, srv519935400=79, srv313685128=66, srv1356543144=15, srv834792470=90, srv1741829077=35, srv1020252823=0, srv1856110441=38, srv2094387873=59, srv2031148266=48, srv123933117=10, srv1761901900=36, srv772931035=89, srv1727896613=34, srv2088752251=58, srv1926685836=41, srv294802281=64, srv1034632116=1, srv134179578=14, srv1260954345=12, srv984399391=99, srv1122203348=6, srv1669570045=29, srv1114514665=5, srv267255830=63, srv457194205=74, srv61713732=83, srv252515376=62, srv155071402=23, srv469253582=76, srv1934188582=42, srv1095996807=4, srv1676230324=30, srv1482670322=20, srv1612804385=27, srv2036089316=50, srv506279912=78, srv315542634=67, srv1969307767=45, srv451023979=73, srv1220173609=9, srv962989538=95, srv1052008619=2, srv967507115=96, srv967767579=97, srv1544157149=22, srv66690928=85, srv211285569=61, srv538287274=80, srv1720967160=32, srv2109044521=60, srv332210952=68, srv391851444=70, srv2032044132=49, srv2058426743=51, srv462520556=75, srv1378533930=17, srv1266535654=13, srv2059596851=52, srv1566678535=24, srv690157299=86, srv739408904=88, srv2085120895=54, srv916486289=94, srv1967577809=44, srv1569304287=25, srv608239356=81, srv969346810=98, srv1980540833=46, srv2029297133=47, srv297475273=65, srv611080261=82, srv639580070=84, srv1660488987=28, srv708601223=87, srv856992338=91, srv2085563553=55, srv1691677296=31, srv1374119815=16, srv1441433420=19, srv2086875034=57, srv1143933894=7, srv1893173979=39, srv1519802789=21, srv1083502833=3, srv1572302990=26, srv382793774=69, srv502215884=77, srv1934474037=43, srv1725937896=33, srv115363311=8, srv2065623202=53, srv904514984=93, srv416194638=72, srv1853777415=37} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-17 23:00:25,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-17 23:00:25,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-17 23:00:25,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-17 23:00:25,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-17 23:00:25,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-17 23:00:25,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-17 23:00:25,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-17 23:00:25,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-17 23:00:25,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-17 23:00:25,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-17 23:00:25,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-17 23:00:25,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-17 23:00:25,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-17 23:00:25,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-17 23:00:25,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-17 23:00:25,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-17 23:00:25,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-17 23:00:25,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-17 23:00:25,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-17 23:00:25,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-17 23:00:25,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-17 23:00:25,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-17 23:00:25,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-17 23:00:25,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-17 23:00:25,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-17 23:00:25,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-17 23:00:25,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-17 23:00:25,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-17 23:00:25,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-17 23:00:25,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-17 23:00:25,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-17 23:00:25,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-17 23:00:25,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-17 23:00:25,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-17 23:00:25,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-17 23:00:25,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-17 23:00:25,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-17 23:00:25,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-17 23:00:25,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-17 23:00:25,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-17 23:00:25,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-17 23:00:25,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-17 23:00:25,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-17 23:00:25,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-17 23:00:25,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-17 23:00:25,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-17 23:00:25,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-17 23:00:25,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-17 23:00:25,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-17 23:00:25,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-17 23:00:25,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-17 23:00:25,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-17 23:00:25,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-17 23:00:25,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-17 23:00:25,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-17 23:00:25,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-17 23:00:25,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-17 23:00:25,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-17 23:00:25,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-17 23:00:25,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-17 23:00:25,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-17 23:00:25,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-17 23:00:25,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-17 23:00:25,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-17 23:00:25,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-17 23:00:25,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-17 23:00:25,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-17 23:00:25,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-17 23:00:25,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-17 23:00:25,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-17 23:00:25,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-17 23:00:25,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-17 23:00:25,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-17 23:00:25,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-17 23:00:25,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-17 23:00:25,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-17 23:00:25,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-17 23:00:25,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-17 23:00:25,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-17 23:00:25,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-17 23:00:25,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-17 23:00:25,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-17 23:00:25,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-17 23:00:25,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-17 23:00:25,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-17 23:00:25,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-17 23:00:25,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-17 23:00:25,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-17 23:00:25,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-17 23:00:25,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-17 23:00:25,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-17 23:00:25,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-17 23:00:25,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-17 23:00:25,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-17 23:00:25,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-17 23:00:25,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-17 23:00:25,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-17 23:00:25,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-17 23:00:25,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-17 23:00:25,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-17 23:00:25,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-17 23:00:25,789 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020597146021777486); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.034135137789004136, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020597146021777486); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043416666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-17 23:00:25,789 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-17 23:00:25,790 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.014964877595567217, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020597146021777486); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.034135137789004136, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020597146021777486); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043416666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-17 23:00:25,963 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 279 ms to try 5000 different iterations.  Found a solution that moves 976 regions; Going from a computed imbalance of 0.014964877595567217 to a new imbalance of 6.466449940068251E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005650148677010918); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.011090890691968327); MoveCostFunction : (multiplier=7.0, imbalance=0.10844444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005650148677010918); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00375); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-17 23:00:25,965 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv2094387873:81 , srv252515376:82 , srv1083502833:83 , srv1266535654:83 , srv1378533930:83 , srv1482670322:83 , srv1725937896:83 , srv1926685836:83 , srv2088752251:83 , srv313685128:83 , srv457194205:83 , srv608239356:83 , srv1020252823:84 , srv1095996807:84 , srv1660488987:84 , srv1669570045:84 , srv1761901900:84 , srv1980540833:84 , srv2036089316:84 , srv2065623202:84 , srv391851444:84 , srv462520556:84 , srv538287274:84 , srv690157299:84 , srv1034632116:85 , srv1114514665:85 , srv1122203348:85 , srv1260954345:85 , srv1356543144:85 , srv1394773989:85 , srv1441433420:85 , srv1720967160:85 , srv1741829077:85 , srv1893173979:85 , srv1934474037:85 , srv1969307767:85 , srv2086302170:85 , srv2109044521:85 , srv315542634:85 , srv382793774:85 , srv502215884:85 , srv506279912:85 , srv639580070:85 , srv66690928:85 , srv708601223:85 , srv772931035:85 , srv856992338:85 , srv861360450:85 , srv962989538:85 , srv967507115:85 , srv984399391:85 , srv1143933894:86 , srv1220173609:86 , srv123933117:86 , srv1254916663:86 , srv134179578:86 , srv1519802789:86 , srv155071402:86 , srv1566678535:86 , srv1569304287:86 , srv1853777415:86 , srv1925290858:86 , srv1967577809:86 , srv2029297133:86 , srv2031148266:86 , srv2032044132:86 , srv2085563553:86 , srv2086875034:86 , srv211285569:86 , srv267255830:86 , srv294802281:86 , srv297475273:86 , srv332210952:86 , srv469253582:86 , srv519935400:86 , srv61713732:86 , srv739408904:86 , srv834792470:86 , srv904514984:86 , srv967767579:86 , srv1052008619:87 , srv1374119815:87 , srv1544157149:87 , srv1572302990:87 , srv1676230324:87 , srv1691677296:87 , srv1856110441:87 , srv2059596851:87 , srv2085120895:87 , srv399787272:87 , srv416194638:87 , srv611080261:87 , srv916486289:87 , srv969346810:87 , srv115363311:88 , srv1727896613:88 , srv1934188582:88 , srv451023979:88 , srv1612804385:89 , srv2058426743:551 }
2023-04-17 23:00:25,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-17 23:00:25,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv861360450=92, srv399787272=71, srv1394773989=18, srv1925290858=40, srv1254916663=11, srv2086302170=56, srv519935400=79, srv313685128=66, srv1356543144=15, srv834792470=90, srv1741829077=35, srv1020252823=0, srv1856110441=38, srv2094387873=59, srv2031148266=48, srv123933117=10, srv1761901900=36, srv772931035=89, srv1727896613=34, srv2088752251=58, srv1926685836=41, srv294802281=64, srv1034632116=1, srv134179578=14, srv1260954345=12, srv984399391=99, srv1122203348=6, srv1669570045=29, srv1114514665=5, srv267255830=63, srv457194205=74, srv61713732=83, srv252515376=62, srv155071402=23, srv469253582=76, srv1934188582=42, srv1095996807=4, srv1676230324=30, srv1482670322=20, srv1612804385=27, srv2036089316=50, srv506279912=78, srv315542634=67, srv1969307767=45, srv451023979=73, srv1220173609=9, srv962989538=95, srv1052008619=2, srv967507115=96, srv967767579=97, srv1544157149=22, srv66690928=85, srv211285569=61, srv538287274=80, srv1720967160=32, srv2109044521=60, srv332210952=68, srv391851444=70, srv2032044132=49, srv2058426743=51, srv462520556=75, srv1378533930=17, srv1266535654=13, srv2059596851=52, srv1566678535=24, srv690157299=86, srv739408904=88, srv2085120895=54, srv916486289=94, srv1967577809=44, srv1569304287=25, srv608239356=81, srv969346810=98, srv1980540833=46, srv2029297133=47, srv297475273=65, srv611080261=82, srv639580070=84, srv1660488987=28, srv708601223=87, srv856992338=91, srv2085563553=55, srv1691677296=31, srv1374119815=16, srv1441433420=19, srv2086875034=57, srv1143933894=7, srv1893173979=39, srv1519802789=21, srv1083502833=3, srv1572302990=26, srv382793774=69, srv502215884=77, srv1934474037=43, srv1725937896=33, srv115363311=8, srv2065623202=53, srv904514984=93, srv416194638=72, srv1853777415=37} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-17 23:00:25,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-17 23:00:25,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-17 23:00:25,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-17 23:00:25,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-17 23:00:25,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-17 23:00:25,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-17 23:00:25,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-17 23:00:25,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-17 23:00:25,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-17 23:00:25,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-17 23:00:25,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-17 23:00:25,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-17 23:00:25,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-17 23:00:25,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-17 23:00:25,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-17 23:00:25,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-17 23:00:25,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-17 23:00:25,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-17 23:00:25,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-17 23:00:25,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-17 23:00:25,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-17 23:00:25,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-17 23:00:25,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-17 23:00:25,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-17 23:00:25,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-17 23:00:25,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-17 23:00:25,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-17 23:00:25,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-17 23:00:25,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-17 23:00:25,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-17 23:00:25,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-17 23:00:25,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-17 23:00:25,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-17 23:00:25,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-17 23:00:25,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-17 23:00:25,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-17 23:00:25,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-17 23:00:25,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-17 23:00:25,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-17 23:00:25,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-17 23:00:25,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-17 23:00:25,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-17 23:00:25,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-17 23:00:25,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-17 23:00:25,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-17 23:00:25,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-17 23:00:25,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-17 23:00:25,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-17 23:00:25,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-17 23:00:25,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-17 23:00:25,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-17 23:00:25,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-17 23:00:25,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-17 23:00:25,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-17 23:00:25,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-17 23:00:25,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-17 23:00:25,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-17 23:00:25,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-17 23:00:25,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-17 23:00:25,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-17 23:00:25,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-17 23:00:25,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-17 23:00:25,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-17 23:00:25,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-17 23:00:25,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-17 23:00:25,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-17 23:00:25,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-17 23:00:25,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-17 23:00:25,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-17 23:00:25,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-17 23:00:25,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-17 23:00:25,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-17 23:00:25,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-17 23:00:25,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-17 23:00:25,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-17 23:00:25,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-17 23:00:25,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-17 23:00:25,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-17 23:00:25,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-17 23:00:25,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-17 23:00:25,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-17 23:00:25,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-17 23:00:25,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-17 23:00:25,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-17 23:00:25,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-17 23:00:25,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-17 23:00:25,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-17 23:00:25,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-17 23:00:25,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-17 23:00:25,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-17 23:00:25,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-17 23:00:25,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-17 23:00:25,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-17 23:00:25,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-17 23:00:25,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-17 23:00:25,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-17 23:00:25,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-17 23:00:25,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-17 23:00:25,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-17 23:00:25,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-17 23:00:25,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-17 23:00:26,070 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005650148677010918); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.011090890691968327); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005650148677010918); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00375); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-17 23:00:26,071 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-17 23:00:26,071 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=6.398108959387182E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005650148677010918); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.011090890691968327); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005650148677010918); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00375); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-17 23:00:26,298 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 304 ms to try 5000 different iterations.  Found a solution that moves 236 regions; Going from a computed imbalance of 6.398108959387182E-4 to a new imbalance of 1.3577033283056557E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0021711298242630987); MoveCostFunction : (multiplier=7.0, imbalance=0.026222222222222223, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-17 23:00:26,298 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1052008619:83 , srv1356543144:83 , srv1925290858:83 , srv451023979:83 , srv856992338:83 , srv1083502833:84 , srv1122203348:84 , srv1220173609:84 , srv123933117:84 , srv1254916663:84 , srv1394773989:84 , srv1441433420:84 , srv1482670322:84 , srv1519802789:84 , srv1544157149:84 , srv155071402:84 , srv1566678535:84 , srv1572302990:84 , srv1612804385:84 , srv1660488987:84 , srv1741829077:84 , srv1853777415:84 , srv1893173979:84 , srv1934188582:84 , srv1934474037:84 , srv1980540833:84 , srv2032044132:84 , srv2059596851:84 , srv2065623202:84 , srv2085120895:84 , srv2086302170:84 , srv2088752251:84 , srv2109044521:84 , srv211285569:84 , srv267255830:84 , srv297475273:84 , srv332210952:84 , srv399787272:84 , srv469253582:84 , srv502215884:84 , srv608239356:84 , srv611080261:84 , srv639580070:84 , srv66690928:84 , srv708601223:84 , srv834792470:84 , srv861360450:84 , srv904514984:84 , srv916486289:84 , srv962989538:84 , srv967507115:84 , srv1020252823:85 , srv1034632116:85 , srv1095996807:85 , srv1114514665:85 , srv1143933894:85 , srv115363311:85 , srv1266535654:85 , srv134179578:85 , srv1374119815:85 , srv1378533930:85 , srv1569304287:85 , srv1669570045:85 , srv1676230324:85 , srv1691677296:85 , srv1720967160:85 , srv1727896613:85 , srv1761901900:85 , srv1856110441:85 , srv1926685836:85 , srv1967577809:85 , srv2029297133:85 , srv2031148266:85 , srv2036089316:85 , srv2085563553:85 , srv2086875034:85 , srv2094387873:85 , srv252515376:85 , srv294802281:85 , srv313685128:85 , srv315542634:85 , srv382793774:85 , srv391851444:85 , srv416194638:85 , srv457194205:85 , srv462520556:85 , srv519935400:85 , srv690157299:85 , srv739408904:85 , srv967767579:85 , srv969346810:85 , srv984399391:85 , srv1260954345:86 , srv1725937896:86 , srv1969307767:86 , srv506279912:86 , srv538287274:86 , srv61713732:86 , srv772931035:86 , srv2058426743:634 }
2023-04-17 23:00:26,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-17 23:00:26,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv861360450=92, srv399787272=71, srv1394773989=18, srv1925290858=40, srv1254916663=11, srv2086302170=56, srv519935400=79, srv313685128=66, srv1356543144=15, srv834792470=90, srv1741829077=35, srv1020252823=0, srv1856110441=38, srv2094387873=59, srv2031148266=48, srv123933117=10, srv1761901900=36, srv772931035=89, srv1727896613=34, srv2088752251=58, srv1926685836=41, srv294802281=64, srv1034632116=1, srv134179578=14, srv1260954345=12, srv984399391=99, srv1122203348=6, srv1669570045=29, srv1114514665=5, srv267255830=63, srv457194205=74, srv61713732=83, srv252515376=62, srv155071402=23, srv469253582=76, srv1934188582=42, srv1095996807=4, srv1676230324=30, srv1482670322=20, srv1612804385=27, srv2036089316=50, srv506279912=78, srv315542634=67, srv1969307767=45, srv451023979=73, srv1220173609=9, srv962989538=95, srv1052008619=2, srv967507115=96, srv967767579=97, srv1544157149=22, srv66690928=85, srv211285569=61, srv538287274=80, srv1720967160=32, srv2109044521=60, srv332210952=68, srv391851444=70, srv2032044132=49, srv2058426743=51, srv462520556=75, srv1378533930=17, srv1266535654=13, srv2059596851=52, srv1566678535=24, srv690157299=86, srv739408904=88, srv2085120895=54, srv916486289=94, srv1967577809=44, srv1569304287=25, srv608239356=81, srv969346810=98, srv1980540833=46, srv2029297133=47, srv297475273=65, srv611080261=82, srv639580070=84, srv1660488987=28, srv708601223=87, srv856992338=91, srv2085563553=55, srv1691677296=31, srv1374119815=16, srv1441433420=19, srv2086875034=57, srv1143933894=7, srv1893173979=39, srv1519802789=21, srv1083502833=3, srv1572302990=26, srv382793774=69, srv502215884=77, srv1934474037=43, srv1725937896=33, srv115363311=8, srv2065623202=53, srv904514984=93, srv416194638=72, srv1853777415=37} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-17 23:00:26,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-17 23:00:26,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-17 23:00:26,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-17 23:00:26,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-17 23:00:26,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-17 23:00:26,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-17 23:00:26,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-17 23:00:26,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-17 23:00:26,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-17 23:00:26,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-17 23:00:26,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-17 23:00:26,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-17 23:00:26,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-17 23:00:26,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-17 23:00:26,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-17 23:00:26,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-17 23:00:26,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-17 23:00:26,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-17 23:00:26,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-17 23:00:26,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-17 23:00:26,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-17 23:00:26,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-17 23:00:26,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-17 23:00:26,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-17 23:00:26,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-17 23:00:26,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-17 23:00:26,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-17 23:00:26,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-17 23:00:26,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-17 23:00:26,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-17 23:00:26,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-17 23:00:26,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-17 23:00:26,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-17 23:00:26,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-17 23:00:26,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-17 23:00:26,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-17 23:00:26,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-17 23:00:26,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-17 23:00:26,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-17 23:00:26,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-17 23:00:26,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-17 23:00:26,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-17 23:00:26,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-17 23:00:26,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-17 23:00:26,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-17 23:00:26,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-17 23:00:26,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-17 23:00:26,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-17 23:00:26,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-17 23:00:26,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-17 23:00:26,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-17 23:00:26,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-17 23:00:26,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-17 23:00:26,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-17 23:00:26,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-17 23:00:26,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-17 23:00:26,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-17 23:00:26,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-17 23:00:26,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-17 23:00:26,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-17 23:00:26,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-17 23:00:26,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-17 23:00:26,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-17 23:00:26,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-17 23:00:26,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-17 23:00:26,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-17 23:00:26,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-17 23:00:26,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-17 23:00:26,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-17 23:00:26,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-17 23:00:26,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-17 23:00:26,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-17 23:00:26,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-17 23:00:26,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-17 23:00:26,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-17 23:00:26,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-17 23:00:26,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-17 23:00:26,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-17 23:00:26,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-17 23:00:26,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-17 23:00:26,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-17 23:00:26,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-17 23:00:26,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-17 23:00:26,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-17 23:00:26,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-17 23:00:26,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-17 23:00:26,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-17 23:00:26,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-17 23:00:26,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-17 23:00:26,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-17 23:00:26,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-17 23:00:26,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-17 23:00:26,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-17 23:00:26,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-17 23:00:26,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-17 23:00:26,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-17 23:00:26,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-17 23:00:26,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-17 23:00:26,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-17 23:00:26,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-17 23:00:26,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-17 23:00:26,398 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.1924525963309394E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0021711298242630987); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-17 23:00:26,398 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1052008619:83 , srv1356543144:83 , srv1925290858:83 , srv451023979:83 , srv856992338:83 , srv1083502833:84 , srv1122203348:84 , srv1220173609:84 , srv123933117:84 , srv1254916663:84 , srv1394773989:84 , srv1441433420:84 , srv1482670322:84 , srv1519802789:84 , srv1544157149:84 , srv155071402:84 , srv1566678535:84 , srv1572302990:84 , srv1612804385:84 , srv1660488987:84 , srv1741829077:84 , srv1853777415:84 , srv1893173979:84 , srv1934188582:84 , srv1934474037:84 , srv1980540833:84 , srv2032044132:84 , srv2059596851:84 , srv2065623202:84 , srv2085120895:84 , srv2086302170:84 , srv2088752251:84 , srv2109044521:84 , srv211285569:84 , srv267255830:84 , srv297475273:84 , srv332210952:84 , srv399787272:84 , srv469253582:84 , srv502215884:84 , srv608239356:84 , srv611080261:84 , srv639580070:84 , srv66690928:84 , srv708601223:84 , srv834792470:84 , srv861360450:84 , srv904514984:84 , srv916486289:84 , srv962989538:84 , srv967507115:84 , srv1020252823:85 , srv1034632116:85 , srv1095996807:85 , srv1114514665:85 , srv1143933894:85 , srv115363311:85 , srv1266535654:85 , srv134179578:85 , srv1374119815:85 , srv1378533930:85 , srv1569304287:85 , srv1669570045:85 , srv1676230324:85 , srv1691677296:85 , srv1720967160:85 , srv1727896613:85 , srv1761901900:85 , srv1856110441:85 , srv1926685836:85 , srv1967577809:85 , srv2029297133:85 , srv2031148266:85 , srv2036089316:85 , srv2085563553:85 , srv2086875034:85 , srv2094387873:85 , srv252515376:85 , srv294802281:85 , srv313685128:85 , srv315542634:85 , srv382793774:85 , srv391851444:85 , srv416194638:85 , srv457194205:85 , srv462520556:85 , srv519935400:85 , srv690157299:85 , srv739408904:85 , srv967767579:85 , srv969346810:85 , srv984399391:85 , srv1260954345:86 , srv1725937896:86 , srv1969307767:86 , srv506279912:86 , srv538287274:86 , srv61713732:86 , srv772931035:86 , srv2058426743:634 }
2023-04-17 23:00:26,418 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=178 (was 150) - SystemLoadAverage LEAK? -, ProcessCount=167 (was 167), AvailableMemoryMB=11888 (was 12398)
2023-04-17 23:00:26,425 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=178, ProcessCount=167, AvailableMemoryMB=11887
2023-04-17 23:00:26,425 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-17 23:00:26,426 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-04-17 23:00:26,437 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1059693421:84 , srv1118737579:84 , srv1307663494:84 , srv1550966727:84 , srv1555894268:84 , srv1771524537:84 , srv1776132281:84 , srv1792278091:84 , srv1824109536:84 , srv1919215432:84 , srv1997985393:84 , srv2032930637:84 , srv2105994057:84 , srv2113245894:84 , srv216916036:84 , srv250548335:84 , srv283013502:84 , srv30329676:84 , srv32237537:84 , srv419989957:84 , srv461425643:84 , srv49119842:84 , srv50363387:84 , srv565399706:84 , srv617124508:84 , srv667072346:84 , srv672306148:84 , srv696398037:84 , srv769035852:84 , srv1608087195:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-04-17 23:00:26,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-17 23:00:26,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv667072346=26, srv50363387=23, srv2113245894=14, srv30329676=18, srv2105994057=13, srv1776132281=7, srv419989957=20, srv1555894268=4, srv2032930637=12, srv32237537=19, srv283013502=17, srv696398037=28, srv1118737579=1, srv1771524537=6, srv1059693421=0, srv617124508=25, srv672306148=27, srv49119842=22, srv1550966727=3, srv1919215432=10, srv216916036=15, srv769035852=29, srv1307663494=2, srv1997985393=11, srv1608087195=5, srv1792278091=8, srv1824109536=9, srv461425643=21, srv250548335=16, srv565399706=24} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-17 23:00:26,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-17 23:00:26,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-17 23:00:26,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-17 23:00:26,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-17 23:00:26,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-17 23:00:26,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-17 23:00:26,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-17 23:00:26,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-17 23:00:26,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-17 23:00:26,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-17 23:00:26,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-17 23:00:26,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-17 23:00:26,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-17 23:00:26,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-17 23:00:26,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-17 23:00:26,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-17 23:00:26,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-17 23:00:26,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-17 23:00:26,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-17 23:00:26,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-17 23:00:26,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-17 23:00:26,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-17 23:00:26,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-17 23:00:26,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-17 23:00:26,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-17 23:00:26,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-17 23:00:26,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-17 23:00:26,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-17 23:00:26,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-17 23:00:26,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-17 23:00:26,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-04-17 23:00:26,477 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1205378646597184, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-17 23:00:26,477 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912572854139899, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1205378646597184, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-04-17 23:02:26,457 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 9351867 different iterations.  Found a solution that moves 1843 regions; Going from a computed imbalance of 0.9912572854139899 to a new imbalance of 4.3016539410932495E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6825925925925926, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-17 23:02:26,462 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1059693421:90 , srv1118737579:90 , srv1307663494:90 , srv1550966727:90 , srv1555894268:90 , srv1608087195:90 , srv1771524537:90 , srv1776132281:90 , srv1792278091:90 , srv1824109536:90 , srv1919215432:90 , srv1997985393:90 , srv2032930637:90 , srv2105994057:90 , srv2113245894:90 , srv216916036:90 , srv250548335:90 , srv283013502:90 , srv30329676:90 , srv32237537:90 , srv419989957:90 , srv461425643:90 , srv49119842:90 , srv50363387:90 , srv565399706:90 , srv617124508:90 , srv667072346:90 , srv672306148:90 , srv696398037:90 , srv769035852:90 }
2023-04-17 23:02:26,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-17 23:02:26,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv667072346=26, srv50363387=23, srv2113245894=14, srv30329676=18, srv2105994057=13, srv1776132281=7, srv419989957=20, srv1555894268=4, srv2032930637=12, srv32237537=19, srv283013502=17, srv696398037=28, srv1118737579=1, srv1771524537=6, srv1059693421=0, srv617124508=25, srv672306148=27, srv49119842=22, srv1550966727=3, srv1919215432=10, srv216916036=15, srv769035852=29, srv1307663494=2, srv1997985393=11, srv1608087195=5, srv1792278091=8, srv1824109536=9, srv461425643=21, srv250548335=16, srv565399706=24} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-17 23:02:26,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-17 23:02:26,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-17 23:02:26,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-17 23:02:26,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-17 23:02:26,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-17 23:02:26,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-17 23:02:26,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-17 23:02:26,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-17 23:02:26,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-17 23:02:26,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-17 23:02:26,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-17 23:02:26,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-17 23:02:26,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-17 23:02:26,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-17 23:02:26,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-17 23:02:26,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-17 23:02:26,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-17 23:02:26,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-17 23:02:26,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-17 23:02:26,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-17 23:02:26,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-17 23:02:26,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-17 23:02:26,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-17 23:02:26,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-17 23:02:26,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-17 23:02:26,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-17 23:02:26,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-17 23:02:26,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-17 23:02:26,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-17 23:02:26,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-17 23:02:26,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-04-17 23:02:26,482 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-17 23:02:26,482 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1059693421:90 , srv1118737579:90 , srv1307663494:90 , srv1550966727:90 , srv1555894268:90 , srv1608087195:90 , srv1771524537:90 , srv1776132281:90 , srv1792278091:90 , srv1824109536:90 , srv1919215432:90 , srv1997985393:90 , srv2032930637:90 , srv2105994057:90 , srv2113245894:90 , srv216916036:90 , srv250548335:90 , srv283013502:90 , srv30329676:90 , srv32237537:90 , srv419989957:90 , srv461425643:90 , srv49119842:90 , srv50363387:90 , srv565399706:90 , srv617124508:90 , srv667072346:90 , srv672306148:90 , srv696398037:90 , srv769035852:90 }
2023-04-17 23:02:26,496 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=200 (was 178) - SystemLoadAverage LEAK? -, ProcessCount=165 (was 167), AvailableMemoryMB=11889 (was 11887) - AvailableMemoryMB LEAK? -