2023-02-17 11:54:01,934 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-02-17 11:54:02,267 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-02-17 11:54:02,378 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-02-17 11:54:02,449 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-17 11:54:02,456 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-02-17 11:54:02,469 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=316, ProcessCount=170, AvailableMemoryMB=8999
2023-02-17 11:54:02,471 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-17 11:54:02,472 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-02-17 11:54:02,601 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1013754018:84 , srv1021837862:84 , srv1049662749:84 , srv1086644892:84 , srv1095082398:84 , srv1121748501:84 , srv1158547014:84 , srv1172230805:84 , srv1174302465:84 , srv1201446047:84 , srv1219212300:84 , srv1220865545:84 , srv1225603987:84 , srv1228354451:84 , srv1234845239:84 , srv1237604818:84 , srv1264094322:84 , srv1270334698:84 , srv1286394355:84 , srv141669808:84 , srv1442414729:84 , srv1487049871:84 , srv1511085940:84 , srv152896981:84 , srv1548089575:84 , srv1556365479:84 , srv1572612559:84 , srv1573474521:84 , srv1580203253:84 , srv1588127499:84 , srv1626347729:84 , srv163633875:84 , srv1639382269:84 , srv1680843615:84 , srv168941730:84 , srv1724978810:84 , srv1740829661:84 , srv1783416116:84 , srv1800644560:84 , srv1800798570:84 , srv1821974687:84 , srv1868375562:84 , srv1917002108:84 , srv1948614439:84 , srv1950646988:84 , srv1954153876:84 , srv1997183132:84 , srv199831729:84 , srv2009614733:84 , srv2013983988:84 , srv2036612317:84 , srv207404754:84 , srv2107128829:84 , srv2109446324:84 , srv2111875227:84 , srv2129750838:84 , srv2134354623:84 , srv2142845784:84 , srv24350806:84 , srv26174103:84 , srv301125708:84 , srv320008430:84 , srv332691477:84 , srv338410459:84 , srv36364878:84 , srv376653572:84 , srv401299:84 , srv402997320:84 , srv424660546:84 , srv432928795:84 , srv465005737:84 , srv469781623:84 , srv477352001:84 , srv516952761:84 , srv565659006:84 , srv566000363:84 , srv579787618:84 , srv579846361:84 , srv588155691:84 , srv595987657:84 , srv596287072:84 , srv624897418:84 , srv662584287:84 , srv684609837:84 , srv688875424:84 , srv740364312:84 , srv741482381:84 , srv74368591:84 , srv749266548:84 , srv760184257:84 , srv774447969:84 , srv793131809:84 , srv811487304:84 , srv832473365:84 , srv836571428:84 , srv853611979:84 , srv89331212:84 , srv937970326:84 , srv944207966:84 , srv856978807:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-02-17 11:54:02,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-17 11:54:02,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1556365479=25, srv1219212300=10, srv469781623=71, srv760184257=89, srv1588127499=29, srv1868375562=41, srv1573474521=27, srv2036612317=50, srv1724978810=35, srv1121748501=5, srv465005737=70, srv1639382269=32, srv2111875227=54, srv1201446047=9, srv1740829661=36, srv1086644892=3, srv1049662749=2, srv1237604818=15, srv2129750838=55, srv1264094322=16, srv811487304=92, srv856978807=96, srv1548089575=24, srv1800798570=39, srv749266548=88, srv684609837=83, srv832473365=93, srv1234845239=14, srv944207966=99, srv588155691=78, srv163633875=31, srv168941730=34, srv1954153876=45, srv477352001=72, srv937970326=98, srv141669808=19, srv624897418=81, srv1013754018=0, srv853611979=95, srv579846361=77, srv432928795=69, srv2013983988=49, srv740364312=85, srv1917002108=42, srv579787618=76, srv595987657=79, srv1286394355=18, srv1158547014=6, srv1511085940=22, srv2142845784=57, srv566000363=75, srv688875424=84, srv376653572=65, srv152896981=23, srv332691477=62, srv1220865545=11, srv1174302465=8, srv1821974687=40, srv1228354451=13, srv401299=66, srv1225603987=12, srv741482381=86, srv1487049871=21, srv338410459=63, srv1270334698=17, srv1572612559=26, srv1948614439=43, srv1095082398=4, srv424660546=68, srv1626347729=30, srv2134354623=56, srv207404754=51, srv516952761=73, srv1680843615=33, srv565659006=74, srv301125708=60, srv1442414729=20, srv402997320=67, srv199831729=47, srv1172230805=7, srv1580203253=28, srv320008430=61, srv2107128829=52, srv26174103=59, srv36364878=64, srv1021837862=1, srv2009614733=48, srv774447969=90, srv1950646988=44, srv24350806=58, srv596287072=80, srv1800644560=38, srv1997183132=46, srv74368591=87, srv89331212=97, srv662584287=82, srv793131809=91, srv836571428=94, srv1783416116=37, srv2109446324=53} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-17 11:54:02,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-17 11:54:02,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-17 11:54:02,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-17 11:54:02,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-17 11:54:02,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-17 11:54:02,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-17 11:54:02,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-17 11:54:02,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-17 11:54:02,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-17 11:54:02,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-17 11:54:02,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-17 11:54:02,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-17 11:54:02,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-17 11:54:02,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-17 11:54:02,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-17 11:54:02,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-17 11:54:02,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-17 11:54:02,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-17 11:54:02,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-17 11:54:02,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-17 11:54:02,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-17 11:54:02,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-17 11:54:02,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-17 11:54:02,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-17 11:54:02,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-17 11:54:02,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-17 11:54:02,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-17 11:54:02,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-17 11:54:02,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-17 11:54:02,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-17 11:54:02,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-17 11:54:02,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-17 11:54:02,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-17 11:54:02,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-17 11:54:02,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-17 11:54:02,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-17 11:54:02,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-17 11:54:02,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-17 11:54:02,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-17 11:54:02,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-17 11:54:02,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-17 11:54:02,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-17 11:54:02,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-17 11:54:02,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-17 11:54:02,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-17 11:54:02,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-17 11:54:02,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-17 11:54:02,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-17 11:54:02,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-17 11:54:02,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-17 11:54:02,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-17 11:54:02,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-17 11:54:02,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-17 11:54:02,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-17 11:54:02,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-17 11:54:02,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-17 11:54:02,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-17 11:54:02,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-17 11:54:02,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-17 11:54:02,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-17 11:54:02,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-17 11:54:02,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-17 11:54:02,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-17 11:54:02,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-17 11:54:02,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-17 11:54:02,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-17 11:54:02,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-17 11:54:02,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-17 11:54:02,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-17 11:54:02,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-17 11:54:02,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-17 11:54:02,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-17 11:54:02,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-17 11:54:02,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-17 11:54:02,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-17 11:54:02,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-17 11:54:02,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-17 11:54:02,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-17 11:54:02,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-17 11:54:02,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-17 11:54:02,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-17 11:54:02,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-17 11:54:02,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-17 11:54:02,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-17 11:54:02,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-17 11:54:02,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-17 11:54:02,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-17 11:54:02,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-17 11:54:02,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-17 11:54:02,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-17 11:54:02,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-17 11:54:02,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-17 11:54:02,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-17 11:54:02,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-17 11:54:02,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-17 11:54:02,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-17 11:54:02,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-17 11:54:02,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-17 11:54:02,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-17 11:54:02,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-17 11:54:02,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-17 11:54:02,970 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-17 11:54:02,970 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-17 11:54:02,970 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-17 11:54:03,297 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 530 ms to try 5000 different iterations.  Found a solution that moves 4695 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17639109367808112. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.037456546715918436, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.058605186641069305, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5216666666666666, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.037456546715918436, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17141666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23983333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-17 11:54:03,333 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1487049871:81 , srv1021837862:82 , srv1172230805:82 , srv338410459:82 , srv1783416116:83 , srv199831729:83 , srv1121748501:84 , srv1228354451:84 , srv1270334698:84 , srv1511085940:84 , srv163633875:84 , srv1917002108:84 , srv2009614733:84 , srv402997320:84 , srv741482381:84 , srv853611979:84 , srv1049662749:85 , srv1225603987:85 , srv1237604818:85 , srv1442414729:85 , srv152896981:85 , srv1821974687:85 , srv1948614439:85 , srv1950646988:85 , srv2013983988:85 , srv2107128829:85 , srv2134354623:85 , srv24350806:85 , srv465005737:85 , srv624897418:85 , srv740364312:85 , srv836571428:85 , srv944207966:85 , srv1158547014:86 , srv1220865545:86 , srv1286394355:86 , srv1573474521:86 , srv1588127499:86 , srv1740829661:86 , srv1800798570:86 , srv207404754:86 , srv2129750838:86 , srv26174103:86 , srv320008430:86 , srv596287072:86 , srv684609837:86 , srv1086644892:87 , srv1201446047:87 , srv1264094322:87 , srv1626347729:87 , srv1639382269:87 , srv168941730:87 , srv1997183132:87 , srv2109446324:87 , srv2142845784:87 , srv332691477:87 , srv36364878:87 , srv376653572:87 , srv424660546:87 , srv432928795:87 , srv516952761:87 , srv565659006:87 , srv566000363:87 , srv749266548:87 , srv832473365:87 , srv1013754018:88 , srv1174302465:88 , srv1219212300:88 , srv1680843615:88 , srv1800644560:88 , srv2036612317:88 , srv301125708:88 , srv477352001:88 , srv579787618:88 , srv688875424:88 , srv74368591:88 , srv793131809:88 , srv811487304:88 , srv89331212:88 , srv937970326:88 , srv1095082398:89 , srv141669808:89 , srv1556365479:89 , srv1572612559:89 , srv1954153876:89 , srv2111875227:89 , srv662584287:89 , srv774447969:89 , srv1234845239:90 , srv1548089575:90 , srv1724978810:90 , srv1868375562:90 , srv401299:90 , srv469781623:90 , srv579846361:90 , srv595987657:90 , srv760184257:91 , srv1580203253:92 , srv588155691:93 , srv856978807:423 }
2023-02-17 11:54:03,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-17 11:54:03,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1556365479=25, srv1219212300=10, srv469781623=71, srv760184257=89, srv1588127499=29, srv1868375562=41, srv1573474521=27, srv2036612317=50, srv1724978810=35, srv1121748501=5, srv465005737=70, srv1639382269=32, srv2111875227=54, srv1201446047=9, srv1740829661=36, srv1086644892=3, srv1049662749=2, srv1237604818=15, srv2129750838=55, srv1264094322=16, srv811487304=92, srv856978807=96, srv1548089575=24, srv1800798570=39, srv749266548=88, srv684609837=83, srv832473365=93, srv1234845239=14, srv944207966=99, srv588155691=78, srv163633875=31, srv168941730=34, srv1954153876=45, srv477352001=72, srv937970326=98, srv141669808=19, srv624897418=81, srv1013754018=0, srv853611979=95, srv579846361=77, srv432928795=69, srv2013983988=49, srv740364312=85, srv1917002108=42, srv579787618=76, srv595987657=79, srv1286394355=18, srv1158547014=6, srv1511085940=22, srv2142845784=57, srv566000363=75, srv688875424=84, srv376653572=65, srv152896981=23, srv332691477=62, srv1220865545=11, srv1174302465=8, srv1821974687=40, srv1228354451=13, srv401299=66, srv1225603987=12, srv741482381=86, srv1487049871=21, srv338410459=63, srv1270334698=17, srv1572612559=26, srv1948614439=43, srv1095082398=4, srv424660546=68, srv1626347729=30, srv2134354623=56, srv207404754=51, srv516952761=73, srv1680843615=33, srv565659006=74, srv301125708=60, srv1442414729=20, srv402997320=67, srv199831729=47, srv1172230805=7, srv1580203253=28, srv320008430=61, srv2107128829=52, srv26174103=59, srv36364878=64, srv1021837862=1, srv2009614733=48, srv774447969=90, srv1950646988=44, srv24350806=58, srv596287072=80, srv1800644560=38, srv1997183132=46, srv74368591=87, srv89331212=97, srv662584287=82, srv793131809=91, srv836571428=94, srv1783416116=37, srv2109446324=53} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-17 11:54:03,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-17 11:54:03,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-17 11:54:03,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-17 11:54:03,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-17 11:54:03,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-17 11:54:03,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-17 11:54:03,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-17 11:54:03,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-17 11:54:03,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-17 11:54:03,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-17 11:54:03,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-17 11:54:03,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-17 11:54:03,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-17 11:54:03,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-17 11:54:03,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-17 11:54:03,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-17 11:54:03,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-17 11:54:03,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-17 11:54:03,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-17 11:54:03,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-17 11:54:03,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-17 11:54:03,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-17 11:54:03,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-17 11:54:03,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-17 11:54:03,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-17 11:54:03,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-17 11:54:03,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-17 11:54:03,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-17 11:54:03,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-17 11:54:03,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-17 11:54:03,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-17 11:54:03,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-17 11:54:03,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-17 11:54:03,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-17 11:54:03,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-17 11:54:03,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-17 11:54:03,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-17 11:54:03,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-17 11:54:03,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-17 11:54:03,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-17 11:54:03,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-17 11:54:03,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-17 11:54:03,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-17 11:54:03,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-17 11:54:03,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-17 11:54:03,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-17 11:54:03,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-17 11:54:03,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-17 11:54:03,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-17 11:54:03,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-17 11:54:03,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-17 11:54:03,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-17 11:54:03,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-17 11:54:03,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-17 11:54:03,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-17 11:54:03,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-17 11:54:03,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-17 11:54:03,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-17 11:54:03,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-17 11:54:03,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-17 11:54:03,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-17 11:54:03,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-17 11:54:03,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-17 11:54:03,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-17 11:54:03,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-17 11:54:03,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-17 11:54:03,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-17 11:54:03,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-17 11:54:03,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-17 11:54:03,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-17 11:54:03,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-17 11:54:03,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-17 11:54:03,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-17 11:54:03,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-17 11:54:03,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-17 11:54:03,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-17 11:54:03,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-17 11:54:03,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-17 11:54:03,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-17 11:54:03,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-17 11:54:03,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-17 11:54:03,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-17 11:54:03,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-17 11:54:03,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-17 11:54:03,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-17 11:54:03,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-17 11:54:03,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-17 11:54:03,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-17 11:54:03,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-17 11:54:03,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-17 11:54:03,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-17 11:54:03,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-17 11:54:03,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-17 11:54:03,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-17 11:54:03,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-17 11:54:03,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-17 11:54:03,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-17 11:54:03,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-17 11:54:03,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-17 11:54:03,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-17 11:54:03,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-17 11:54:03,518 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.037456546715918436, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.058605186641069305, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.037456546715918436, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17141666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23983333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-17 11:54:03,518 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-17 11:54:03,518 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17635821858542766, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.037456546715918436, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.058605186641069305, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.037456546715918436, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17141666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23983333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-17 11:54:03,825 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 438 ms to try 5000 different iterations.  Found a solution that moves 2693 regions; Going from a computed imbalance of 0.17635821858542766 to a new imbalance of 0.014442259307850107. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020481788954164575); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03563438231739869, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2992222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020481788954164575); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011083333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0465, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-17 11:54:03,839 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv579846361:80 , srv1556365479:81 , srv1580203253:81 , srv588155691:81 , srv760184257:81 , srv1548089575:82 , srv1724978810:82 , srv1954153876:82 , srv2111875227:82 , srv401299:82 , srv579787618:82 , srv595987657:82 , srv1174302465:83 , srv1234845239:83 , srv141669808:83 , srv1680843615:83 , srv301125708:83 , srv469781623:83 , srv565659006:83 , srv688875424:83 , srv774447969:83 , srv832473365:83 , srv937970326:83 , srv1013754018:84 , srv1095082398:84 , srv1219212300:84 , srv1868375562:84 , srv432928795:84 , srv662584287:84 , srv74368591:84 , srv811487304:84 , srv1049662749:85 , srv1220865545:85 , srv1237604818:85 , srv1573474521:85 , srv1800798570:85 , srv2013983988:85 , srv2142845784:85 , srv320008430:85 , srv332691477:85 , srv36364878:85 , srv376653572:85 , srv596287072:85 , srv749266548:85 , srv793131809:85 , srv89331212:85 , srv1086644892:86 , srv1158547014:86 , srv1572612559:86 , srv1626347729:86 , srv163633875:86 , srv1639382269:86 , srv168941730:86 , srv1800644560:86 , srv1821974687:86 , srv1950646988:86 , srv1997183132:86 , srv2036612317:86 , srv2107128829:86 , srv2129750838:86 , srv24350806:86 , srv424660546:86 , srv477352001:86 , srv516952761:86 , srv566000363:86 , srv624897418:86 , srv836571428:86 , srv1121748501:87 , srv1225603987:87 , srv1286394355:87 , srv1442414729:87 , srv1588127499:87 , srv1948614439:87 , srv2009614733:87 , srv2109446324:87 , srv2134354623:87 , srv465005737:87 , srv684609837:87 , srv740364312:87 , srv944207966:87 , srv1201446047:88 , srv1487049871:88 , srv1740829661:88 , srv1917002108:88 , srv207404754:88 , srv26174103:88 , srv741482381:88 , srv1172230805:89 , srv1228354451:89 , srv1511085940:89 , srv199831729:89 , srv402997320:89 , srv853611979:89 , srv1264094322:90 , srv1270334698:90 , srv152896981:90 , srv1783416116:90 , srv1021837862:91 , srv338410459:91 , srv856978807:533 }
2023-02-17 11:54:03,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-17 11:54:03,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1556365479=25, srv1219212300=10, srv469781623=71, srv760184257=89, srv1588127499=29, srv1868375562=41, srv1573474521=27, srv2036612317=50, srv1724978810=35, srv1121748501=5, srv465005737=70, srv1639382269=32, srv2111875227=54, srv1201446047=9, srv1740829661=36, srv1086644892=3, srv1049662749=2, srv1237604818=15, srv2129750838=55, srv1264094322=16, srv811487304=92, srv856978807=96, srv1548089575=24, srv1800798570=39, srv749266548=88, srv684609837=83, srv832473365=93, srv1234845239=14, srv944207966=99, srv588155691=78, srv163633875=31, srv168941730=34, srv1954153876=45, srv477352001=72, srv937970326=98, srv141669808=19, srv624897418=81, srv1013754018=0, srv853611979=95, srv579846361=77, srv432928795=69, srv2013983988=49, srv740364312=85, srv1917002108=42, srv579787618=76, srv595987657=79, srv1286394355=18, srv1158547014=6, srv1511085940=22, srv2142845784=57, srv566000363=75, srv688875424=84, srv376653572=65, srv152896981=23, srv332691477=62, srv1220865545=11, srv1174302465=8, srv1821974687=40, srv1228354451=13, srv401299=66, srv1225603987=12, srv741482381=86, srv1487049871=21, srv338410459=63, srv1270334698=17, srv1572612559=26, srv1948614439=43, srv1095082398=4, srv424660546=68, srv1626347729=30, srv2134354623=56, srv207404754=51, srv516952761=73, srv1680843615=33, srv565659006=74, srv301125708=60, srv1442414729=20, srv402997320=67, srv199831729=47, srv1172230805=7, srv1580203253=28, srv320008430=61, srv2107128829=52, srv26174103=59, srv36364878=64, srv1021837862=1, srv2009614733=48, srv774447969=90, srv1950646988=44, srv24350806=58, srv596287072=80, srv1800644560=38, srv1997183132=46, srv74368591=87, srv89331212=97, srv662584287=82, srv793131809=91, srv836571428=94, srv1783416116=37, srv2109446324=53} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-17 11:54:03,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-17 11:54:03,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-17 11:54:03,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-17 11:54:03,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-17 11:54:03,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-17 11:54:03,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-17 11:54:03,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-17 11:54:03,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-17 11:54:03,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-17 11:54:03,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-17 11:54:03,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-17 11:54:03,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-17 11:54:03,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-17 11:54:03,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-17 11:54:03,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-17 11:54:03,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-17 11:54:03,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-17 11:54:03,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-17 11:54:03,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-17 11:54:03,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-17 11:54:03,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-17 11:54:03,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-17 11:54:03,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-17 11:54:03,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-17 11:54:03,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-17 11:54:03,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-17 11:54:03,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-17 11:54:03,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-17 11:54:03,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-17 11:54:03,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-17 11:54:03,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-17 11:54:03,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-17 11:54:03,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-17 11:54:03,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-17 11:54:03,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-17 11:54:03,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-17 11:54:03,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-17 11:54:03,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-17 11:54:03,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-17 11:54:03,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-17 11:54:03,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-17 11:54:03,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-17 11:54:03,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-17 11:54:03,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-17 11:54:03,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-17 11:54:03,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-17 11:54:03,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-17 11:54:03,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-17 11:54:03,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-17 11:54:03,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-17 11:54:03,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-17 11:54:03,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-17 11:54:03,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-17 11:54:03,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-17 11:54:03,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-17 11:54:03,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-17 11:54:03,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-17 11:54:03,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-17 11:54:03,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-17 11:54:03,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-17 11:54:03,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-17 11:54:03,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-17 11:54:03,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-17 11:54:03,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-17 11:54:03,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-17 11:54:03,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-17 11:54:03,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-17 11:54:03,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-17 11:54:03,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-17 11:54:03,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-17 11:54:03,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-17 11:54:03,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-17 11:54:03,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-17 11:54:03,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-17 11:54:03,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-17 11:54:03,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-17 11:54:03,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-17 11:54:03,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-17 11:54:03,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-17 11:54:03,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-17 11:54:03,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-17 11:54:03,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-17 11:54:03,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-17 11:54:03,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-17 11:54:03,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-17 11:54:03,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-17 11:54:03,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-17 11:54:03,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-17 11:54:03,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-17 11:54:03,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-17 11:54:03,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-17 11:54:03,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-17 11:54:03,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-17 11:54:03,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-17 11:54:03,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-17 11:54:03,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-17 11:54:03,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-17 11:54:03,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-17 11:54:03,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-17 11:54:03,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-17 11:54:03,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-17 11:54:04,010 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020481788954164575); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03563438231739869, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020481788954164575); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011083333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0465, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-17 11:54:04,010 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-17 11:54:04,011 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.014423402518815874, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020481788954164575); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03563438231739869, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020481788954164575); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011083333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0465, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-17 11:54:04,259 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 363 ms to try 5000 different iterations.  Found a solution that moves 1012 regions; Going from a computed imbalance of 0.014423402518815874 to a new imbalance of 7.585513780077705E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006243590053822763); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.010260359250125632); MoveCostFunction : (multiplier=7.0, imbalance=0.11244444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006243590053822763); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0033333333333333335); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-17 11:54:04,262 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1264094322:80 , srv1572612559:82 , srv1286394355:83 , srv152896981:83 , srv1868375562:83 , srv1095082398:84 , srv1158547014:84 , srv1201446047:84 , srv1219212300:84 , srv1228354451:84 , srv1234845239:84 , srv1270334698:84 , srv1588127499:84 , srv1626347729:84 , srv168941730:84 , srv1724978810:84 , srv1740829661:84 , srv1783416116:84 , srv1800644560:84 , srv1917002108:84 , srv2009614733:84 , srv2036612317:84 , srv207404754:84 , srv26174103:84 , srv477352001:84 , srv588155691:84 , srv793131809:84 , srv89331212:84 , srv1013754018:85 , srv1021837862:85 , srv1086644892:85 , srv1172230805:85 , srv141669808:85 , srv1511085940:85 , srv1548089575:85 , srv1580203253:85 , srv1948614439:85 , srv1997183132:85 , srv199831729:85 , srv2109446324:85 , srv2129750838:85 , srv2134354623:85 , srv2142845784:85 , srv24350806:85 , srv332691477:85 , srv338410459:85 , srv36364878:85 , srv376653572:85 , srv401299:85 , srv402997320:85 , srv424660546:85 , srv469781623:85 , srv516952761:85 , srv566000363:85 , srv595987657:85 , srv596287072:85 , srv662584287:85 , srv740364312:85 , srv74368591:85 , srv749266548:85 , srv853611979:85 , srv1049662749:86 , srv1121748501:86 , srv1225603987:86 , srv1442414729:86 , srv1573474521:86 , srv1639382269:86 , srv1800798570:86 , srv1821974687:86 , srv1950646988:86 , srv2111875227:86 , srv301125708:86 , srv465005737:86 , srv741482381:86 , srv811487304:86 , srv836571428:86 , srv937970326:86 , srv1174302465:87 , srv1220865545:87 , srv1237604818:87 , srv1556365479:87 , srv163633875:87 , srv1680843615:87 , srv1954153876:87 , srv2013983988:87 , srv2107128829:87 , srv320008430:87 , srv579846361:87 , srv624897418:87 , srv684609837:87 , srv760184257:87 , srv774447969:87 , srv832473365:87 , srv944207966:87 , srv1487049871:88 , srv432928795:88 , srv565659006:88 , srv579787618:88 , srv688875424:88 , srv856978807:557 }
2023-02-17 11:54:04,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-17 11:54:04,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1556365479=25, srv1219212300=10, srv469781623=71, srv760184257=89, srv1588127499=29, srv1868375562=41, srv1573474521=27, srv2036612317=50, srv1724978810=35, srv1121748501=5, srv465005737=70, srv1639382269=32, srv2111875227=54, srv1201446047=9, srv1740829661=36, srv1086644892=3, srv1049662749=2, srv1237604818=15, srv2129750838=55, srv1264094322=16, srv811487304=92, srv856978807=96, srv1548089575=24, srv1800798570=39, srv749266548=88, srv684609837=83, srv832473365=93, srv1234845239=14, srv944207966=99, srv588155691=78, srv163633875=31, srv168941730=34, srv1954153876=45, srv477352001=72, srv937970326=98, srv141669808=19, srv624897418=81, srv1013754018=0, srv853611979=95, srv579846361=77, srv432928795=69, srv2013983988=49, srv740364312=85, srv1917002108=42, srv579787618=76, srv595987657=79, srv1286394355=18, srv1158547014=6, srv1511085940=22, srv2142845784=57, srv566000363=75, srv688875424=84, srv376653572=65, srv152896981=23, srv332691477=62, srv1220865545=11, srv1174302465=8, srv1821974687=40, srv1228354451=13, srv401299=66, srv1225603987=12, srv741482381=86, srv1487049871=21, srv338410459=63, srv1270334698=17, srv1572612559=26, srv1948614439=43, srv1095082398=4, srv424660546=68, srv1626347729=30, srv2134354623=56, srv207404754=51, srv516952761=73, srv1680843615=33, srv565659006=74, srv301125708=60, srv1442414729=20, srv402997320=67, srv199831729=47, srv1172230805=7, srv1580203253=28, srv320008430=61, srv2107128829=52, srv26174103=59, srv36364878=64, srv1021837862=1, srv2009614733=48, srv774447969=90, srv1950646988=44, srv24350806=58, srv596287072=80, srv1800644560=38, srv1997183132=46, srv74368591=87, srv89331212=97, srv662584287=82, srv793131809=91, srv836571428=94, srv1783416116=37, srv2109446324=53} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-17 11:54:04,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-17 11:54:04,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-17 11:54:04,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-17 11:54:04,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-17 11:54:04,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-17 11:54:04,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-17 11:54:04,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-17 11:54:04,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-17 11:54:04,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-17 11:54:04,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-17 11:54:04,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-17 11:54:04,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-17 11:54:04,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-17 11:54:04,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-17 11:54:04,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-17 11:54:04,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-17 11:54:04,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-17 11:54:04,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-17 11:54:04,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-17 11:54:04,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-17 11:54:04,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-17 11:54:04,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-17 11:54:04,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-17 11:54:04,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-17 11:54:04,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-17 11:54:04,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-17 11:54:04,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-17 11:54:04,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-17 11:54:04,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-17 11:54:04,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-17 11:54:04,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-17 11:54:04,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-17 11:54:04,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-17 11:54:04,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-17 11:54:04,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-17 11:54:04,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-17 11:54:04,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-17 11:54:04,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-17 11:54:04,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-17 11:54:04,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-17 11:54:04,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-17 11:54:04,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-17 11:54:04,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-17 11:54:04,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-17 11:54:04,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-17 11:54:04,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-17 11:54:04,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-17 11:54:04,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-17 11:54:04,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-17 11:54:04,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-17 11:54:04,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-17 11:54:04,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-17 11:54:04,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-17 11:54:04,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-17 11:54:04,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-17 11:54:04,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-17 11:54:04,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-17 11:54:04,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-17 11:54:04,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-17 11:54:04,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-17 11:54:04,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-17 11:54:04,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-17 11:54:04,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-17 11:54:04,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-17 11:54:04,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-17 11:54:04,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-17 11:54:04,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-17 11:54:04,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-17 11:54:04,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-17 11:54:04,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-17 11:54:04,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-17 11:54:04,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-17 11:54:04,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-17 11:54:04,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-17 11:54:04,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-17 11:54:04,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-17 11:54:04,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-17 11:54:04,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-17 11:54:04,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-17 11:54:04,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-17 11:54:04,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-17 11:54:04,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-17 11:54:04,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-17 11:54:04,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-17 11:54:04,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-17 11:54:04,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-17 11:54:04,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-17 11:54:04,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-17 11:54:04,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-17 11:54:04,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-17 11:54:04,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-17 11:54:04,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-17 11:54:04,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-17 11:54:04,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-17 11:54:04,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-17 11:54:04,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-17 11:54:04,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-17 11:54:04,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-17 11:54:04,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-17 11:54:04,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-17 11:54:04,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-17 11:54:04,408 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006243590053822763); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.010260359250125632); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006243590053822763); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0033333333333333335); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-17 11:54:04,408 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-17 11:54:04,408 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=7.514652025519055E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006243590053822763); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.010260359250125632); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006243590053822763); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0033333333333333335); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-17 11:54:04,749 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 444 ms to try 5000 different iterations.  Found a solution that moves 290 regions; Going from a computed imbalance of 7.514652025519055E-4 to a new imbalance of 5.3279501232150236E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.868392354571063E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0026378918903435544); MoveCostFunction : (multiplier=7.0, imbalance=0.03222222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.868392354571063E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=4.166666666666667E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-17 11:54:04,750 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1201446047:83 , srv1580203253:83 , srv1639382269:83 , srv1740829661:83 , srv2109446324:83 , srv432928795:83 , srv684609837:83 , srv688875424:83 , srv774447969:83 , srv944207966:83 , srv1013754018:84 , srv1021837862:84 , srv1086644892:84 , srv1174302465:84 , srv1225603987:84 , srv1270334698:84 , srv1442414729:84 , srv1511085940:84 , srv152896981:84 , srv163633875:84 , srv1800644560:84 , srv1954153876:84 , srv1997183132:84 , srv207404754:84 , srv2107128829:84 , srv2129750838:84 , srv26174103:84 , srv320008430:84 , srv338410459:84 , srv402997320:84 , srv424660546:84 , srv465005737:84 , srv469781623:84 , srv477352001:84 , srv516952761:84 , srv565659006:84 , srv566000363:84 , srv579787618:84 , srv588155691:84 , srv624897418:84 , srv662584287:84 , srv741482381:84 , srv749266548:84 , srv760184257:84 , srv811487304:84 , srv853611979:84 , srv1095082398:85 , srv1121748501:85 , srv1220865545:85 , srv1228354451:85 , srv1234845239:85 , srv1237604818:85 , srv1264094322:85 , srv141669808:85 , srv1487049871:85 , srv1548089575:85 , srv1556365479:85 , srv1572612559:85 , srv1573474521:85 , srv1588127499:85 , srv1626347729:85 , srv1680843615:85 , srv168941730:85 , srv1783416116:85 , srv1800798570:85 , srv1821974687:85 , srv1868375562:85 , srv1948614439:85 , srv1950646988:85 , srv199831729:85 , srv2013983988:85 , srv2036612317:85 , srv2111875227:85 , srv2134354623:85 , srv2142845784:85 , srv301125708:85 , srv332691477:85 , srv36364878:85 , srv376653572:85 , srv401299:85 , srv579846361:85 , srv595987657:85 , srv740364312:85 , srv793131809:85 , srv832473365:85 , srv836571428:85 , srv89331212:85 , srv937970326:85 , srv1049662749:86 , srv1158547014:86 , srv1172230805:86 , srv1219212300:86 , srv1286394355:86 , srv1724978810:86 , srv1917002108:86 , srv24350806:86 , srv596287072:86 , srv74368591:86 , srv2009614733:87 , srv856978807:629 }
2023-02-17 11:54:04,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-17 11:54:04,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1556365479=25, srv1219212300=10, srv469781623=71, srv760184257=89, srv1588127499=29, srv1868375562=41, srv1573474521=27, srv2036612317=50, srv1724978810=35, srv1121748501=5, srv465005737=70, srv1639382269=32, srv2111875227=54, srv1201446047=9, srv1740829661=36, srv1086644892=3, srv1049662749=2, srv1237604818=15, srv2129750838=55, srv1264094322=16, srv811487304=92, srv856978807=96, srv1548089575=24, srv1800798570=39, srv749266548=88, srv684609837=83, srv832473365=93, srv1234845239=14, srv944207966=99, srv588155691=78, srv163633875=31, srv168941730=34, srv1954153876=45, srv477352001=72, srv937970326=98, srv141669808=19, srv624897418=81, srv1013754018=0, srv853611979=95, srv579846361=77, srv432928795=69, srv2013983988=49, srv740364312=85, srv1917002108=42, srv579787618=76, srv595987657=79, srv1286394355=18, srv1158547014=6, srv1511085940=22, srv2142845784=57, srv566000363=75, srv688875424=84, srv376653572=65, srv152896981=23, srv332691477=62, srv1220865545=11, srv1174302465=8, srv1821974687=40, srv1228354451=13, srv401299=66, srv1225603987=12, srv741482381=86, srv1487049871=21, srv338410459=63, srv1270334698=17, srv1572612559=26, srv1948614439=43, srv1095082398=4, srv424660546=68, srv1626347729=30, srv2134354623=56, srv207404754=51, srv516952761=73, srv1680843615=33, srv565659006=74, srv301125708=60, srv1442414729=20, srv402997320=67, srv199831729=47, srv1172230805=7, srv1580203253=28, srv320008430=61, srv2107128829=52, srv26174103=59, srv36364878=64, srv1021837862=1, srv2009614733=48, srv774447969=90, srv1950646988=44, srv24350806=58, srv596287072=80, srv1800644560=38, srv1997183132=46, srv74368591=87, srv89331212=97, srv662584287=82, srv793131809=91, srv836571428=94, srv1783416116=37, srv2109446324=53} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-17 11:54:04,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-17 11:54:04,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-17 11:54:04,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-17 11:54:04,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-17 11:54:04,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-17 11:54:04,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-17 11:54:04,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-17 11:54:04,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-17 11:54:04,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-17 11:54:04,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-17 11:54:04,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-17 11:54:04,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-17 11:54:04,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-17 11:54:04,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-17 11:54:04,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-17 11:54:04,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-17 11:54:04,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-17 11:54:04,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-17 11:54:04,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-17 11:54:04,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-17 11:54:04,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-17 11:54:04,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-17 11:54:04,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-17 11:54:04,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-17 11:54:04,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-17 11:54:04,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-17 11:54:04,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-17 11:54:04,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-17 11:54:04,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-17 11:54:04,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-17 11:54:04,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-17 11:54:04,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-17 11:54:04,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-17 11:54:04,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-17 11:54:04,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-17 11:54:04,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-17 11:54:04,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-17 11:54:04,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-17 11:54:04,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-17 11:54:04,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-17 11:54:04,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-17 11:54:04,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-17 11:54:04,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-17 11:54:04,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-17 11:54:04,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-17 11:54:04,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-17 11:54:04,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-17 11:54:04,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-17 11:54:04,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-17 11:54:04,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-17 11:54:04,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-17 11:54:04,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-17 11:54:04,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-17 11:54:04,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-17 11:54:04,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-17 11:54:04,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-17 11:54:04,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-17 11:54:04,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-17 11:54:04,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-17 11:54:04,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-17 11:54:04,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-17 11:54:04,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-17 11:54:04,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-17 11:54:04,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-17 11:54:04,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-17 11:54:04,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-17 11:54:04,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-17 11:54:04,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-17 11:54:04,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-17 11:54:04,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-17 11:54:04,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-17 11:54:04,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-17 11:54:04,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-17 11:54:04,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-17 11:54:04,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-17 11:54:04,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-17 11:54:04,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-17 11:54:04,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-17 11:54:04,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-17 11:54:04,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-17 11:54:04,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-17 11:54:04,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-17 11:54:04,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-17 11:54:04,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-17 11:54:04,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-17 11:54:04,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-17 11:54:04,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-17 11:54:04,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-17 11:54:04,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-17 11:54:04,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-17 11:54:04,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-17 11:54:04,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-17 11:54:04,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-17 11:54:04,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-17 11:54:04,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-17 11:54:04,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-17 11:54:04,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-17 11:54:04,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-17 11:54:04,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-17 11:54:04,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-17 11:54:04,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-17 11:54:04,883 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.868392354571063E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0026378918903435544); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.868392354571063E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=4.166666666666667E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-17 11:54:04,884 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-17 11:54:04,884 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=5.124887783076601E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.868392354571063E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0026378918903435544); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.868392354571063E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=4.166666666666667E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-17 11:54:05,115 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 328 ms to try 5000 different iterations.  Found a solution that moves 50 regions; Going from a computed imbalance of 5.124887783076601E-5 to a new imbalance of 9.143297722316076E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0019534423844256277); MoveCostFunction : (multiplier=7.0, imbalance=0.005555555555555556); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-17 11:54:05,116 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1220865545:83 , srv1680843615:83 , srv2036612317:83 , srv684609837:83 , srv74368591:83 , srv760184257:83 , srv1021837862:84 , srv1049662749:84 , srv1086644892:84 , srv1095082398:84 , srv1121748501:84 , srv1158547014:84 , srv1172230805:84 , srv1174302465:84 , srv1201446047:84 , srv1219212300:84 , srv1225603987:84 , srv1228354451:84 , srv1234845239:84 , srv1237604818:84 , srv1264094322:84 , srv1270334698:84 , srv1286394355:84 , srv141669808:84 , srv1442414729:84 , srv1487049871:84 , srv1511085940:84 , srv152896981:84 , srv1548089575:84 , srv1556365479:84 , srv1572612559:84 , srv1573474521:84 , srv1588127499:84 , srv1626347729:84 , srv1639382269:84 , srv168941730:84 , srv1724978810:84 , srv1783416116:84 , srv1800644560:84 , srv1800798570:84 , srv1821974687:84 , srv1868375562:84 , srv1917002108:84 , srv1948614439:84 , srv1950646988:84 , srv1954153876:84 , srv1997183132:84 , srv199831729:84 , srv2009614733:84 , srv2013983988:84 , srv207404754:84 , srv2107128829:84 , srv2109446324:84 , srv2111875227:84 , srv2134354623:84 , srv2142845784:84 , srv24350806:84 , srv26174103:84 , srv301125708:84 , srv320008430:84 , srv332691477:84 , srv338410459:84 , srv36364878:84 , srv376653572:84 , srv401299:84 , srv402997320:84 , srv424660546:84 , srv432928795:84 , srv465005737:84 , srv469781623:84 , srv477352001:84 , srv516952761:84 , srv565659006:84 , srv566000363:84 , srv579787618:84 , srv579846361:84 , srv588155691:84 , srv595987657:84 , srv596287072:84 , srv624897418:84 , srv662584287:84 , srv688875424:84 , srv740364312:84 , srv741482381:84 , srv774447969:84 , srv793131809:84 , srv811487304:84 , srv832473365:84 , srv836571428:84 , srv853611979:84 , srv89331212:84 , srv937970326:84 , srv944207966:84 , srv1013754018:85 , srv1580203253:85 , srv163633875:85 , srv1740829661:85 , srv2129750838:85 , srv749266548:85 , srv856978807:684 }
2023-02-17 11:54:05,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-17 11:54:05,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1556365479=25, srv1219212300=10, srv469781623=71, srv760184257=89, srv1588127499=29, srv1868375562=41, srv1573474521=27, srv2036612317=50, srv1724978810=35, srv1121748501=5, srv465005737=70, srv1639382269=32, srv2111875227=54, srv1201446047=9, srv1740829661=36, srv1086644892=3, srv1049662749=2, srv1237604818=15, srv2129750838=55, srv1264094322=16, srv811487304=92, srv856978807=96, srv1548089575=24, srv1800798570=39, srv749266548=88, srv684609837=83, srv832473365=93, srv1234845239=14, srv944207966=99, srv588155691=78, srv163633875=31, srv168941730=34, srv1954153876=45, srv477352001=72, srv937970326=98, srv141669808=19, srv624897418=81, srv1013754018=0, srv853611979=95, srv579846361=77, srv432928795=69, srv2013983988=49, srv740364312=85, srv1917002108=42, srv579787618=76, srv595987657=79, srv1286394355=18, srv1158547014=6, srv1511085940=22, srv2142845784=57, srv566000363=75, srv688875424=84, srv376653572=65, srv152896981=23, srv332691477=62, srv1220865545=11, srv1174302465=8, srv1821974687=40, srv1228354451=13, srv401299=66, srv1225603987=12, srv741482381=86, srv1487049871=21, srv338410459=63, srv1270334698=17, srv1572612559=26, srv1948614439=43, srv1095082398=4, srv424660546=68, srv1626347729=30, srv2134354623=56, srv207404754=51, srv516952761=73, srv1680843615=33, srv565659006=74, srv301125708=60, srv1442414729=20, srv402997320=67, srv199831729=47, srv1172230805=7, srv1580203253=28, srv320008430=61, srv2107128829=52, srv26174103=59, srv36364878=64, srv1021837862=1, srv2009614733=48, srv774447969=90, srv1950646988=44, srv24350806=58, srv596287072=80, srv1800644560=38, srv1997183132=46, srv74368591=87, srv89331212=97, srv662584287=82, srv793131809=91, srv836571428=94, srv1783416116=37, srv2109446324=53} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-17 11:54:05,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-17 11:54:05,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-17 11:54:05,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-17 11:54:05,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-17 11:54:05,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-17 11:54:05,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-17 11:54:05,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-17 11:54:05,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-17 11:54:05,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-17 11:54:05,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-17 11:54:05,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-17 11:54:05,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-17 11:54:05,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-17 11:54:05,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-17 11:54:05,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-17 11:54:05,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-17 11:54:05,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-17 11:54:05,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-17 11:54:05,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-17 11:54:05,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-17 11:54:05,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-17 11:54:05,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-17 11:54:05,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-17 11:54:05,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-17 11:54:05,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-17 11:54:05,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-17 11:54:05,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-17 11:54:05,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-17 11:54:05,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-17 11:54:05,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-17 11:54:05,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-17 11:54:05,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-17 11:54:05,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-17 11:54:05,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-17 11:54:05,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-17 11:54:05,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-17 11:54:05,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-17 11:54:05,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-17 11:54:05,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-17 11:54:05,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-17 11:54:05,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-17 11:54:05,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-17 11:54:05,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-17 11:54:05,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-17 11:54:05,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-17 11:54:05,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-17 11:54:05,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-17 11:54:05,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-17 11:54:05,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-17 11:54:05,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-17 11:54:05,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-17 11:54:05,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-17 11:54:05,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-17 11:54:05,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-17 11:54:05,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-17 11:54:05,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-17 11:54:05,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-17 11:54:05,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-17 11:54:05,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-17 11:54:05,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-17 11:54:05,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-17 11:54:05,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-17 11:54:05,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-17 11:54:05,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-17 11:54:05,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-17 11:54:05,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-17 11:54:05,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-17 11:54:05,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-17 11:54:05,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-17 11:54:05,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-17 11:54:05,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-17 11:54:05,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-17 11:54:05,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-17 11:54:05,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-17 11:54:05,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-17 11:54:05,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-17 11:54:05,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-17 11:54:05,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-17 11:54:05,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-17 11:54:05,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-17 11:54:05,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-17 11:54:05,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-17 11:54:05,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-17 11:54:05,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-17 11:54:05,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-17 11:54:05,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-17 11:54:05,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-17 11:54:05,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-17 11:54:05,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-17 11:54:05,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-17 11:54:05,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-17 11:54:05,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-17 11:54:05,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-17 11:54:05,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-17 11:54:05,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-17 11:54:05,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-17 11:54:05,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-17 11:54:05,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-17 11:54:05,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-17 11:54:05,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-17 11:54:05,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-17 11:54:05,253 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=8.793190239318795E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0019534423844256277); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-17 11:54:05,254 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1220865545:83 , srv1680843615:83 , srv2036612317:83 , srv684609837:83 , srv74368591:83 , srv760184257:83 , srv1021837862:84 , srv1049662749:84 , srv1086644892:84 , srv1095082398:84 , srv1121748501:84 , srv1158547014:84 , srv1172230805:84 , srv1174302465:84 , srv1201446047:84 , srv1219212300:84 , srv1225603987:84 , srv1228354451:84 , srv1234845239:84 , srv1237604818:84 , srv1264094322:84 , srv1270334698:84 , srv1286394355:84 , srv141669808:84 , srv1442414729:84 , srv1487049871:84 , srv1511085940:84 , srv152896981:84 , srv1548089575:84 , srv1556365479:84 , srv1572612559:84 , srv1573474521:84 , srv1588127499:84 , srv1626347729:84 , srv1639382269:84 , srv168941730:84 , srv1724978810:84 , srv1783416116:84 , srv1800644560:84 , srv1800798570:84 , srv1821974687:84 , srv1868375562:84 , srv1917002108:84 , srv1948614439:84 , srv1950646988:84 , srv1954153876:84 , srv1997183132:84 , srv199831729:84 , srv2009614733:84 , srv2013983988:84 , srv207404754:84 , srv2107128829:84 , srv2109446324:84 , srv2111875227:84 , srv2134354623:84 , srv2142845784:84 , srv24350806:84 , srv26174103:84 , srv301125708:84 , srv320008430:84 , srv332691477:84 , srv338410459:84 , srv36364878:84 , srv376653572:84 , srv401299:84 , srv402997320:84 , srv424660546:84 , srv432928795:84 , srv465005737:84 , srv469781623:84 , srv477352001:84 , srv516952761:84 , srv565659006:84 , srv566000363:84 , srv579787618:84 , srv579846361:84 , srv588155691:84 , srv595987657:84 , srv596287072:84 , srv624897418:84 , srv662584287:84 , srv688875424:84 , srv740364312:84 , srv741482381:84 , srv774447969:84 , srv793131809:84 , srv811487304:84 , srv832473365:84 , srv836571428:84 , srv853611979:84 , srv89331212:84 , srv937970326:84 , srv944207966:84 , srv1013754018:85 , srv1580203253:85 , srv163633875:85 , srv1740829661:85 , srv2129750838:85 , srv749266548:85 , srv856978807:684 }
2023-02-17 11:54:05,279 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=323 (was 316) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=8485 (was 8999)
2023-02-17 11:54:05,286 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=323, ProcessCount=170, AvailableMemoryMB=8485
2023-02-17 11:54:05,287 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-17 11:54:05,287 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-02-17 11:54:05,296 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1009612927:84 , srv1126060457:84 , srv1229282831:84 , srv1289984247:84 , srv1399282355:84 , srv142967444:84 , srv1502835932:84 , srv1543264837:84 , srv16199650:84 , srv1691506013:84 , srv1839031155:84 , srv1859080722:84 , srv18647353:84 , srv1879449052:84 , srv1945836143:84 , srv2004038101:84 , srv2055869975:84 , srv2057861023:84 , srv2078492100:84 , srv2101583537:84 , srv22400081:84 , srv28470880:84 , srv360394960:84 , srv389159190:84 , srv459851728:84 , srv655591805:84 , srv760757018:84 , srv766115880:84 , srv890902088:84 , srv253340549:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-02-17 11:54:05,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-17 11:54:05,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1229282831=2, srv760757018=27, srv890902088=29, srv22400081=20, srv1126060457=1, srv1859080722=11, srv1839031155=10, srv1009612927=0, srv1399282355=4, srv18647353=12, srv1502835932=6, srv1945836143=14, srv2078492100=18, srv2101583537=19, srv1543264837=7, srv1691506013=9, srv2004038101=15, srv2055869975=16, srv28470880=22, srv655591805=26, srv766115880=28, srv142967444=5, srv459851728=25, srv1879449052=13, srv360394960=23, srv2057861023=17, srv253340549=21, srv389159190=24, srv1289984247=3, srv16199650=8} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-17 11:54:05,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-17 11:54:05,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-17 11:54:05,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-17 11:54:05,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-17 11:54:05,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-17 11:54:05,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-17 11:54:05,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-17 11:54:05,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-17 11:54:05,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-17 11:54:05,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-17 11:54:05,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-17 11:54:05,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-17 11:54:05,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-17 11:54:05,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-17 11:54:05,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-17 11:54:05,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-17 11:54:05,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-17 11:54:05,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-17 11:54:05,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-17 11:54:05,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-17 11:54:05,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-17 11:54:05,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-17 11:54:05,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-17 11:54:05,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-17 11:54:05,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-17 11:54:05,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-17 11:54:05,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-17 11:54:05,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-17 11:54:05,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-17 11:54:05,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-17 11:54:05,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-02-17 11:54:05,322 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0957037955298852, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-17 11:54:05,323 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912494602798979, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0957037955298852, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-02-17 11:56:05,310 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6338231 different iterations.  Found a solution that moves 1853 regions; Going from a computed imbalance of 0.9912494602798979 to a new imbalance of 4.324994439959734E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6862962962962963, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-17 11:56:05,317 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1009612927:90 , srv1126060457:90 , srv1229282831:90 , srv1289984247:90 , srv1399282355:90 , srv142967444:90 , srv1502835932:90 , srv1543264837:90 , srv16199650:90 , srv1691506013:90 , srv1839031155:90 , srv1859080722:90 , srv18647353:90 , srv1879449052:90 , srv1945836143:90 , srv2004038101:90 , srv2055869975:90 , srv2057861023:90 , srv2078492100:90 , srv2101583537:90 , srv22400081:90 , srv253340549:90 , srv28470880:90 , srv360394960:90 , srv389159190:90 , srv459851728:90 , srv655591805:90 , srv760757018:90 , srv766115880:90 , srv890902088:90 }
2023-02-17 11:56:05,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-17 11:56:05,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1229282831=2, srv760757018=27, srv890902088=29, srv22400081=20, srv1126060457=1, srv1859080722=11, srv1839031155=10, srv1009612927=0, srv1399282355=4, srv18647353=12, srv1502835932=6, srv1945836143=14, srv2078492100=18, srv2101583537=19, srv1543264837=7, srv1691506013=9, srv2004038101=15, srv2055869975=16, srv28470880=22, srv655591805=26, srv766115880=28, srv142967444=5, srv459851728=25, srv1879449052=13, srv360394960=23, srv2057861023=17, srv253340549=21, srv389159190=24, srv1289984247=3, srv16199650=8} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-17 11:56:05,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-17 11:56:05,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-17 11:56:05,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-17 11:56:05,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-17 11:56:05,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-17 11:56:05,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-17 11:56:05,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-17 11:56:05,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-17 11:56:05,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-17 11:56:05,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-17 11:56:05,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-17 11:56:05,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-17 11:56:05,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-17 11:56:05,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-17 11:56:05,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-17 11:56:05,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-17 11:56:05,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-17 11:56:05,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-17 11:56:05,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-17 11:56:05,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-17 11:56:05,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-17 11:56:05,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-17 11:56:05,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-17 11:56:05,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-17 11:56:05,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-17 11:56:05,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-17 11:56:05,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-17 11:56:05,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-17 11:56:05,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-17 11:56:05,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-17 11:56:05,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-02-17 11:56:05,342 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-17 11:56:05,342 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1009612927:90 , srv1126060457:90 , srv1229282831:90 , srv1289984247:90 , srv1399282355:90 , srv142967444:90 , srv1502835932:90 , srv1543264837:90 , srv16199650:90 , srv1691506013:90 , srv1839031155:90 , srv1859080722:90 , srv18647353:90 , srv1879449052:90 , srv1945836143:90 , srv2004038101:90 , srv2055869975:90 , srv2057861023:90 , srv2078492100:90 , srv2101583537:90 , srv22400081:90 , srv253340549:90 , srv28470880:90 , srv360394960:90 , srv389159190:90 , srv459851728:90 , srv655591805:90 , srv760757018:90 , srv766115880:90 , srv890902088:90 }
2023-02-17 11:56:05,366 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=191 (was 323), ProcessCount=170 (was 170), AvailableMemoryMB=8300 (was 8485)