2023-02-20 16:54:03,150 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-02-20 16:54:03,496 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-02-20 16:54:03,604 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-02-20 16:54:03,666 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-20 16:54:03,676 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-02-20 16:54:03,691 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=295, ProcessCount=172, AvailableMemoryMB=6552
2023-02-20 16:54:03,694 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-20 16:54:03,694 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-02-20 16:54:03,822 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1018590555:84 , srv1030556872:84 , srv1030991426:84 , srv1033242031:84 , srv1049181296:84 , srv105945959:84 , srv109725926:84 , srv1100656900:84 , srv1119756568:84 , srv1139071086:84 , srv1141216837:84 , srv114230355:84 , srv1143167504:84 , srv1152278107:84 , srv1167403821:84 , srv1174205876:84 , srv1178625889:84 , srv1241283483:84 , srv1304157763:84 , srv1342715220:84 , srv1359079743:84 , srv1381114967:84 , srv1418381078:84 , srv1418727504:84 , srv1438438151:84 , srv1451658494:84 , srv1466460354:84 , srv1486366799:84 , srv1559312092:84 , srv1566427366:84 , srv1573830714:84 , srv1583609186:84 , srv1604736175:84 , srv1615486875:84 , srv1616165665:84 , srv1620046101:84 , srv1649791449:84 , srv1690779500:84 , srv1798060972:84 , srv183265193:84 , srv1844679285:84 , srv1848428869:84 , srv1912824088:84 , srv1926810950:84 , srv1976608334:84 , srv198825882:84 , srv2007429464:84 , srv2023303331:84 , srv2047294721:84 , srv2057310639:84 , srv2065336459:84 , srv21169281:84 , srv212306007:84 , srv2130533118:84 , srv2138536062:84 , srv217705839:84 , srv225445869:84 , srv225986913:84 , srv227571571:84 , srv235809209:84 , srv28388002:84 , srv284056510:84 , srv29461319:84 , srv332312699:84 , srv356355511:84 , srv370505536:84 , srv398596228:84 , srv413788193:84 , srv424000485:84 , srv438875110:84 , srv466565767:84 , srv486293368:84 , srv54100212:84 , srv542656975:84 , srv56606498:84 , srv574684638:84 , srv607349064:84 , srv671950351:84 , srv703324142:84 , srv705883907:84 , srv736994020:84 , srv745220237:84 , srv756653661:84 , srv766139592:84 , srv76933551:84 , srv795108973:84 , srv798234041:84 , srv828773578:84 , srv840550686:84 , srv844729809:84 , srv860157645:84 , srv900352575:84 , srv901003933:84 , srv902992818:84 , srv904591973:84 , srv92723160:84 , srv929095186:84 , srv953906927:84 , srv971314618:84 , srv15594995:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-02-20 16:54:03,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-20 16:54:03,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv736994020=81, srv901003933=93, srv1690779500=38, srv2007429464=47, srv114230355=11, srv1141216837=10, srv1912824088=43, srv840550686=89, srv1926810950=44, srv756653661=83, srv225986913=58, srv28388002=61, srv466565767=71, srv1486366799=27, srv1649791449=37, srv1018590555=0, srv795108973=86, srv953906927=98, srv212306007=53, srv424000485=69, srv1152278107=13, srv54100212=73, srv332312699=64, srv76933551=85, srv1848428869=42, srv607349064=77, srv486293368=72, srv828773578=88, srv356355511=65, srv574684638=76, srv1143167504=12, srv1466460354=26, srv15594995=29, srv29461319=63, srv1033242031=3, srv198825882=46, srv1573830714=31, srv929095186=97, srv703324142=79, srv798234041=87, srv1178625889=16, srv1359079743=20, srv109725926=6, srv235809209=60, srv1418381078=22, srv1438438151=24, srv217705839=56, srv1241283483=17, srv1030991426=2, srv1030556872=1, srv1566427366=30, srv284056510=62, srv1381114967=21, srv225445869=57, srv1583609186=32, srv1559312092=28, srv1798060972=39, srv1418727504=23, srv1451658494=25, srv705883907=80, srv2130533118=54, srv370505536=66, srv92723160=96, srv1604736175=33, srv1100656900=7, srv1139071086=9, srv2065336459=51, srv2057310639=50, srv413788193=68, srv227571571=59, srv1342715220=19, srv1615486875=34, srv904591973=95, srv1304157763=18, srv438875110=70, srv542656975=74, srv2023303331=48, srv1616165665=35, srv1167403821=14, srv1844679285=41, srv745220237=82, srv1174205876=15, srv2138536062=55, srv844729809=90, srv900352575=92, srv1049181296=4, srv2047294721=49, srv671950351=78, srv183265193=40, srv21169281=52, srv1976608334=45, srv1119756568=8, srv860157645=91, srv902992818=94, srv56606498=75, srv398596228=67, srv105945959=5, srv971314618=99, srv1620046101=36, srv766139592=84} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-20 16:54:03,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-20 16:54:03,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-20 16:54:03,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-20 16:54:03,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-20 16:54:03,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-20 16:54:03,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-20 16:54:03,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-20 16:54:03,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-20 16:54:03,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-20 16:54:03,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-20 16:54:03,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-20 16:54:03,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-20 16:54:03,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-20 16:54:03,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-20 16:54:03,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-20 16:54:03,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-20 16:54:03,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-20 16:54:03,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-20 16:54:03,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-20 16:54:03,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-20 16:54:03,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-20 16:54:03,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-20 16:54:03,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-20 16:54:03,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-20 16:54:03,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-20 16:54:03,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-20 16:54:03,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-20 16:54:03,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-20 16:54:03,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-20 16:54:03,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-20 16:54:03,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-20 16:54:03,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-20 16:54:03,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-20 16:54:03,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-20 16:54:03,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-20 16:54:03,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-20 16:54:03,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-20 16:54:03,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-20 16:54:03,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-20 16:54:03,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-20 16:54:03,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-20 16:54:03,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-20 16:54:03,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-20 16:54:03,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-20 16:54:03,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-20 16:54:03,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-20 16:54:03,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-20 16:54:03,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-20 16:54:03,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-20 16:54:03,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-20 16:54:03,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-20 16:54:03,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-20 16:54:03,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-20 16:54:03,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-20 16:54:03,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-20 16:54:03,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-20 16:54:03,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-20 16:54:03,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-20 16:54:03,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-20 16:54:03,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-20 16:54:03,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-20 16:54:03,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-20 16:54:03,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-20 16:54:03,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-20 16:54:03,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-20 16:54:03,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-20 16:54:03,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-20 16:54:03,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-20 16:54:03,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-20 16:54:03,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-20 16:54:03,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-20 16:54:03,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-20 16:54:03,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-20 16:54:03,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-20 16:54:03,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-20 16:54:03,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-20 16:54:03,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-20 16:54:03,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-20 16:54:03,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-20 16:54:03,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-20 16:54:03,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-20 16:54:03,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-20 16:54:03,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-20 16:54:03,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-20 16:54:03,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-20 16:54:03,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-20 16:54:03,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-20 16:54:03,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-20 16:54:03,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-20 16:54:03,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-20 16:54:03,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-20 16:54:03,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-20 16:54:03,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-20 16:54:03,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-20 16:54:03,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-20 16:54:03,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-20 16:54:03,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-20 16:54:03,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-20 16:54:03,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-20 16:54:03,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-20 16:54:03,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-20 16:54:04,113 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-20 16:54:04,113 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-20 16:54:04,113 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-20 16:54:04,401 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 442 ms to try 5000 different iterations.  Found a solution that moves 4696 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17536989672956338. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.042407003071014346, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05977698921788473, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5217777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.042407003071014346, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1705, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23733333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-20 16:54:04,427 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1100656900:81 , srv1848428869:81 , srv413788193:81 , srv703324142:81 , srv745220237:81 , srv756653661:81 , srv1178625889:82 , srv1926810950:82 , srv2065336459:82 , srv212306007:82 , srv1304157763:83 , srv1583609186:83 , srv332312699:83 , srv705883907:83 , srv798234041:83 , srv1690779500:84 , srv2023303331:84 , srv217705839:84 , srv398596228:84 , srv486293368:84 , srv542656975:84 , srv766139592:84 , srv795108973:84 , srv1049181296:85 , srv1152278107:85 , srv1359079743:85 , srv1649791449:85 , srv1798060972:85 , srv183265193:85 , srv1976608334:85 , srv227571571:85 , srv607349064:85 , srv671950351:85 , srv953906927:85 , srv1033242031:86 , srv1167403821:86 , srv1418381078:86 , srv1418727504:86 , srv1566427366:86 , srv1620046101:86 , srv1844679285:86 , srv1912824088:86 , srv198825882:86 , srv2057310639:86 , srv2130533118:86 , srv356355511:86 , srv370505536:86 , srv466565767:86 , srv574684638:86 , srv840550686:86 , srv844729809:86 , srv902992818:86 , srv92723160:86 , srv1030556872:87 , srv1119756568:87 , srv114230355:87 , srv1174205876:87 , srv1438438151:87 , srv1466460354:87 , srv1559312092:87 , srv1573830714:87 , srv1615486875:87 , srv1616165665:87 , srv2007429464:87 , srv2138536062:87 , srv225445869:87 , srv284056510:87 , srv424000485:87 , srv76933551:87 , srv828773578:87 , srv904591973:87 , srv929095186:87 , srv1018590555:88 , srv1030991426:88 , srv1139071086:88 , srv1141216837:88 , srv1604736175:88 , srv225986913:88 , srv28388002:88 , srv29461319:88 , srv438875110:88 , srv971314618:88 , srv105945959:89 , srv109725926:89 , srv1143167504:89 , srv1241283483:89 , srv235809209:89 , srv900352575:89 , srv1486366799:90 , srv2047294721:90 , srv54100212:90 , srv56606498:90 , srv736994020:90 , srv860157645:90 , srv901003933:90 , srv1342715220:91 , srv1381114967:91 , srv1451658494:92 , srv21169281:92 , srv15594995:467 }
2023-02-20 16:54:04,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-20 16:54:04,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv736994020=81, srv901003933=93, srv1690779500=38, srv2007429464=47, srv114230355=11, srv1141216837=10, srv1912824088=43, srv840550686=89, srv1926810950=44, srv756653661=83, srv225986913=58, srv28388002=61, srv466565767=71, srv1486366799=27, srv1649791449=37, srv1018590555=0, srv795108973=86, srv953906927=98, srv212306007=53, srv424000485=69, srv1152278107=13, srv54100212=73, srv332312699=64, srv76933551=85, srv1848428869=42, srv607349064=77, srv486293368=72, srv828773578=88, srv356355511=65, srv574684638=76, srv1143167504=12, srv1466460354=26, srv15594995=29, srv29461319=63, srv1033242031=3, srv198825882=46, srv1573830714=31, srv929095186=97, srv703324142=79, srv798234041=87, srv1178625889=16, srv1359079743=20, srv109725926=6, srv235809209=60, srv1418381078=22, srv1438438151=24, srv217705839=56, srv1241283483=17, srv1030991426=2, srv1030556872=1, srv1566427366=30, srv284056510=62, srv1381114967=21, srv225445869=57, srv1583609186=32, srv1559312092=28, srv1798060972=39, srv1418727504=23, srv1451658494=25, srv705883907=80, srv2130533118=54, srv370505536=66, srv92723160=96, srv1604736175=33, srv1100656900=7, srv1139071086=9, srv2065336459=51, srv2057310639=50, srv413788193=68, srv227571571=59, srv1342715220=19, srv1615486875=34, srv904591973=95, srv1304157763=18, srv438875110=70, srv542656975=74, srv2023303331=48, srv1616165665=35, srv1167403821=14, srv1844679285=41, srv745220237=82, srv1174205876=15, srv2138536062=55, srv844729809=90, srv900352575=92, srv1049181296=4, srv2047294721=49, srv671950351=78, srv183265193=40, srv21169281=52, srv1976608334=45, srv1119756568=8, srv860157645=91, srv902992818=94, srv56606498=75, srv398596228=67, srv105945959=5, srv971314618=99, srv1620046101=36, srv766139592=84} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-20 16:54:04,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-20 16:54:04,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-20 16:54:04,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-20 16:54:04,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-20 16:54:04,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-20 16:54:04,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-20 16:54:04,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-20 16:54:04,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-20 16:54:04,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-20 16:54:04,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-20 16:54:04,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-20 16:54:04,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-20 16:54:04,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-20 16:54:04,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-20 16:54:04,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-20 16:54:04,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-20 16:54:04,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-20 16:54:04,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-20 16:54:04,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-20 16:54:04,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-20 16:54:04,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-20 16:54:04,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-20 16:54:04,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-20 16:54:04,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-20 16:54:04,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-20 16:54:04,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-20 16:54:04,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-20 16:54:04,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-20 16:54:04,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-20 16:54:04,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-20 16:54:04,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-20 16:54:04,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-20 16:54:04,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-20 16:54:04,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-20 16:54:04,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-20 16:54:04,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-20 16:54:04,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-20 16:54:04,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-20 16:54:04,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-20 16:54:04,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-20 16:54:04,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-20 16:54:04,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-20 16:54:04,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-20 16:54:04,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-20 16:54:04,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-20 16:54:04,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-20 16:54:04,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-20 16:54:04,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-20 16:54:04,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-20 16:54:04,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-20 16:54:04,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-20 16:54:04,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-20 16:54:04,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-20 16:54:04,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-20 16:54:04,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-20 16:54:04,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-20 16:54:04,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-20 16:54:04,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-20 16:54:04,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-20 16:54:04,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-20 16:54:04,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-20 16:54:04,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-20 16:54:04,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-20 16:54:04,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-20 16:54:04,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-20 16:54:04,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-20 16:54:04,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-20 16:54:04,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-20 16:54:04,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-20 16:54:04,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-20 16:54:04,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-20 16:54:04,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-20 16:54:04,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-20 16:54:04,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-20 16:54:04,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-20 16:54:04,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-20 16:54:04,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-20 16:54:04,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-20 16:54:04,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-20 16:54:04,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-20 16:54:04,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-20 16:54:04,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-20 16:54:04,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-20 16:54:04,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-20 16:54:04,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-20 16:54:04,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-20 16:54:04,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-20 16:54:04,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-20 16:54:04,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-20 16:54:04,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-20 16:54:04,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-20 16:54:04,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-20 16:54:04,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-20 16:54:04,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-20 16:54:04,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-20 16:54:04,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-20 16:54:04,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-20 16:54:04,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-20 16:54:04,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-20 16:54:04,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-20 16:54:04,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-20 16:54:04,631 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.042407003071014346, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05977698921788473, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.042407003071014346, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1705, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23733333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-20 16:54:04,632 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-20 16:54:04,632 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17533701463476029, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.042407003071014346, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05977698921788473, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.042407003071014346, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1705, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23733333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-20 16:54:04,937 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 452 ms to try 5000 different iterations.  Found a solution that moves 2564 regions; Going from a computed imbalance of 0.17533701463476029 to a new imbalance of 0.015681975677631688. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023295624491820308); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03989886203789767, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2848888888888889, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023295624491820308); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0125); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04575, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-20 16:54:04,958 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1342715220:79 , srv21169281:79 , srv1381114967:80 , srv1451658494:80 , srv900352575:80 , srv56606498:81 , srv860157645:81 , srv1030991426:82 , srv105945959:82 , srv1559312092:82 , srv438875110:82 , srv1018590555:83 , srv1030556872:83 , srv1119756568:83 , srv1141216837:83 , srv1143167504:83 , srv1486366799:83 , srv2047294721:83 , srv225986913:83 , srv54100212:83 , srv904591973:83 , srv1033242031:84 , srv1174205876:84 , srv1466460354:84 , srv1566427366:84 , srv1573830714:84 , srv1616165665:84 , srv2007429464:84 , srv2057310639:84 , srv235809209:84 , srv29461319:84 , srv424000485:84 , srv574684638:84 , srv736994020:84 , srv76933551:84 , srv109725926:85 , srv1139071086:85 , srv114230355:85 , srv1167403821:85 , srv1418727504:85 , srv1438438151:85 , srv1615486875:85 , srv1798060972:85 , srv1976608334:85 , srv2138536062:85 , srv284056510:85 , srv901003933:85 , srv929095186:85 , srv971314618:85 , srv1049181296:86 , srv1241283483:86 , srv1359079743:86 , srv1418381078:86 , srv1604736175:86 , srv1844679285:86 , srv1912824088:86 , srv198825882:86 , srv2130533118:86 , srv466565767:86 , srv486293368:86 , srv607349064:86 , srv828773578:86 , srv840550686:86 , srv844729809:86 , srv902992818:86 , srv92723160:86 , srv1152278107:87 , srv1690779500:87 , srv356355511:87 , srv370505536:87 , srv398596228:87 , srv1649791449:88 , srv183265193:88 , srv217705839:88 , srv227571571:88 , srv671950351:88 , srv766139592:88 , srv795108973:88 , srv798234041:88 , srv953906927:88 , srv1178625889:89 , srv1583609186:89 , srv2023303331:89 , srv2065336459:89 , srv212306007:89 , srv225445869:89 , srv28388002:89 , srv413788193:89 , srv1304157763:90 , srv332312699:90 , srv542656975:90 , srv756653661:90 , srv1620046101:91 , srv1848428869:91 , srv1926810950:91 , srv703324142:91 , srv1100656900:92 , srv745220237:92 , srv705883907:93 , srv15594995:514 }
2023-02-20 16:54:04,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-20 16:54:04,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv736994020=81, srv901003933=93, srv1690779500=38, srv2007429464=47, srv114230355=11, srv1141216837=10, srv1912824088=43, srv840550686=89, srv1926810950=44, srv756653661=83, srv225986913=58, srv28388002=61, srv466565767=71, srv1486366799=27, srv1649791449=37, srv1018590555=0, srv795108973=86, srv953906927=98, srv212306007=53, srv424000485=69, srv1152278107=13, srv54100212=73, srv332312699=64, srv76933551=85, srv1848428869=42, srv607349064=77, srv486293368=72, srv828773578=88, srv356355511=65, srv574684638=76, srv1143167504=12, srv1466460354=26, srv15594995=29, srv29461319=63, srv1033242031=3, srv198825882=46, srv1573830714=31, srv929095186=97, srv703324142=79, srv798234041=87, srv1178625889=16, srv1359079743=20, srv109725926=6, srv235809209=60, srv1418381078=22, srv1438438151=24, srv217705839=56, srv1241283483=17, srv1030991426=2, srv1030556872=1, srv1566427366=30, srv284056510=62, srv1381114967=21, srv225445869=57, srv1583609186=32, srv1559312092=28, srv1798060972=39, srv1418727504=23, srv1451658494=25, srv705883907=80, srv2130533118=54, srv370505536=66, srv92723160=96, srv1604736175=33, srv1100656900=7, srv1139071086=9, srv2065336459=51, srv2057310639=50, srv413788193=68, srv227571571=59, srv1342715220=19, srv1615486875=34, srv904591973=95, srv1304157763=18, srv438875110=70, srv542656975=74, srv2023303331=48, srv1616165665=35, srv1167403821=14, srv1844679285=41, srv745220237=82, srv1174205876=15, srv2138536062=55, srv844729809=90, srv900352575=92, srv1049181296=4, srv2047294721=49, srv671950351=78, srv183265193=40, srv21169281=52, srv1976608334=45, srv1119756568=8, srv860157645=91, srv902992818=94, srv56606498=75, srv398596228=67, srv105945959=5, srv971314618=99, srv1620046101=36, srv766139592=84} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-20 16:54:04,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-20 16:54:04,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-20 16:54:04,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-20 16:54:04,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-20 16:54:04,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-20 16:54:04,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-20 16:54:04,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-20 16:54:04,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-20 16:54:04,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-20 16:54:04,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-20 16:54:04,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-20 16:54:04,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-20 16:54:04,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-20 16:54:04,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-20 16:54:04,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-20 16:54:04,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-20 16:54:04,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-20 16:54:04,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-20 16:54:04,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-20 16:54:04,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-20 16:54:04,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-20 16:54:04,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-20 16:54:04,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-20 16:54:04,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-20 16:54:04,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-20 16:54:04,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-20 16:54:04,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-20 16:54:04,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-20 16:54:04,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-20 16:54:04,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-20 16:54:04,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-20 16:54:04,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-20 16:54:04,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-20 16:54:04,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-20 16:54:04,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-20 16:54:04,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-20 16:54:04,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-20 16:54:04,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-20 16:54:04,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-20 16:54:04,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-20 16:54:04,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-20 16:54:04,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-20 16:54:04,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-20 16:54:04,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-20 16:54:04,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-20 16:54:04,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-20 16:54:04,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-20 16:54:04,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-20 16:54:04,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-20 16:54:04,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-20 16:54:04,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-20 16:54:04,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-20 16:54:04,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-20 16:54:04,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-20 16:54:04,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-20 16:54:04,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-20 16:54:04,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-20 16:54:04,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-20 16:54:04,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-20 16:54:04,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-20 16:54:04,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-20 16:54:04,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-20 16:54:04,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-20 16:54:04,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-20 16:54:04,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-20 16:54:04,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-20 16:54:04,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-20 16:54:04,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-20 16:54:04,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-20 16:54:04,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-20 16:54:04,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-20 16:54:04,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-20 16:54:04,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-20 16:54:04,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-20 16:54:04,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-20 16:54:04,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-20 16:54:04,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-20 16:54:04,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-20 16:54:04,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-20 16:54:04,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-20 16:54:04,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-20 16:54:04,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-20 16:54:04,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-20 16:54:04,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-20 16:54:04,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-20 16:54:04,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-20 16:54:04,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-20 16:54:04,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-20 16:54:04,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-20 16:54:04,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-20 16:54:04,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-20 16:54:04,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-20 16:54:04,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-20 16:54:04,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-20 16:54:04,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-20 16:54:04,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-20 16:54:04,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-20 16:54:04,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-20 16:54:04,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-20 16:54:04,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-20 16:54:04,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-20 16:54:05,124 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023295624491820308); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03989886203789767, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023295624491820308); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0125); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04575, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-20 16:54:05,124 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-20 16:54:05,124 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.015664022165903586, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023295624491820308); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03989886203789767, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023295624491820308); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0125); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04575, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-20 16:54:05,394 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 379 ms to try 5000 different iterations.  Found a solution that moves 986 regions; Going from a computed imbalance of 0.015664022165903586 to a new imbalance of 8.588994616722018E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008334268559567502); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.012023172538477988); MoveCostFunction : (multiplier=7.0, imbalance=0.10955555555555556, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.008334268559567502); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.005916666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-20 16:54:05,396 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1620046101:80 , srv28388002:80 , srv705883907:81 , srv1241283483:82 , srv225445869:82 , srv901003933:82 , srv1604736175:83 , srv542656975:83 , srv736994020:83 , srv109725926:84 , srv1100656900:84 , srv1139071086:84 , srv1143167504:84 , srv1438438151:84 , srv1451658494:84 , srv1583609186:84 , srv183265193:84 , srv2047294721:84 , srv227571571:84 , srv235809209:84 , srv332312699:84 , srv356355511:84 , srv370505536:84 , srv828773578:84 , srv971314618:84 , srv114230355:85 , srv1152278107:85 , srv1174205876:85 , srv1304157763:85 , srv1486366799:85 , srv1615486875:85 , srv1649791449:85 , srv1844679285:85 , srv1848428869:85 , srv1926810950:85 , srv2023303331:85 , srv2130533118:85 , srv2138536062:85 , srv217705839:85 , srv29461319:85 , srv398596228:85 , srv466565767:85 , srv54100212:85 , srv607349064:85 , srv671950351:85 , srv703324142:85 , srv745220237:85 , srv795108973:85 , srv840550686:85 , srv929095186:85 , srv953906927:85 , srv1049181296:86 , srv105945959:86 , srv1167403821:86 , srv1359079743:86 , srv1381114967:86 , srv1418381078:86 , srv1418727504:86 , srv1466460354:86 , srv1573830714:86 , srv1616165665:86 , srv1690779500:86 , srv1912824088:86 , srv198825882:86 , srv2007429464:86 , srv21169281:86 , srv212306007:86 , srv225986913:86 , srv284056510:86 , srv424000485:86 , srv56606498:86 , srv766139592:86 , srv844729809:86 , srv860157645:86 , srv902992818:86 , srv92723160:86 , srv1018590555:87 , srv1030556872:87 , srv1119756568:87 , srv1141216837:87 , srv1178625889:87 , srv1342715220:87 , srv1798060972:87 , srv1976608334:87 , srv2065336459:87 , srv438875110:87 , srv486293368:87 , srv574684638:87 , srv756653661:87 , srv798234041:87 , srv900352575:87 , srv904591973:87 , srv1030991426:88 , srv1033242031:88 , srv1559312092:88 , srv1566427366:88 , srv2057310639:88 , srv76933551:88 , srv413788193:89 , srv15594995:551 }
2023-02-20 16:54:05,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-20 16:54:05,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv736994020=81, srv901003933=93, srv1690779500=38, srv2007429464=47, srv114230355=11, srv1141216837=10, srv1912824088=43, srv840550686=89, srv1926810950=44, srv756653661=83, srv225986913=58, srv28388002=61, srv466565767=71, srv1486366799=27, srv1649791449=37, srv1018590555=0, srv795108973=86, srv953906927=98, srv212306007=53, srv424000485=69, srv1152278107=13, srv54100212=73, srv332312699=64, srv76933551=85, srv1848428869=42, srv607349064=77, srv486293368=72, srv828773578=88, srv356355511=65, srv574684638=76, srv1143167504=12, srv1466460354=26, srv15594995=29, srv29461319=63, srv1033242031=3, srv198825882=46, srv1573830714=31, srv929095186=97, srv703324142=79, srv798234041=87, srv1178625889=16, srv1359079743=20, srv109725926=6, srv235809209=60, srv1418381078=22, srv1438438151=24, srv217705839=56, srv1241283483=17, srv1030991426=2, srv1030556872=1, srv1566427366=30, srv284056510=62, srv1381114967=21, srv225445869=57, srv1583609186=32, srv1559312092=28, srv1798060972=39, srv1418727504=23, srv1451658494=25, srv705883907=80, srv2130533118=54, srv370505536=66, srv92723160=96, srv1604736175=33, srv1100656900=7, srv1139071086=9, srv2065336459=51, srv2057310639=50, srv413788193=68, srv227571571=59, srv1342715220=19, srv1615486875=34, srv904591973=95, srv1304157763=18, srv438875110=70, srv542656975=74, srv2023303331=48, srv1616165665=35, srv1167403821=14, srv1844679285=41, srv745220237=82, srv1174205876=15, srv2138536062=55, srv844729809=90, srv900352575=92, srv1049181296=4, srv2047294721=49, srv671950351=78, srv183265193=40, srv21169281=52, srv1976608334=45, srv1119756568=8, srv860157645=91, srv902992818=94, srv56606498=75, srv398596228=67, srv105945959=5, srv971314618=99, srv1620046101=36, srv766139592=84} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-20 16:54:05,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-20 16:54:05,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-20 16:54:05,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-20 16:54:05,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-20 16:54:05,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-20 16:54:05,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-20 16:54:05,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-20 16:54:05,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-20 16:54:05,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-20 16:54:05,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-20 16:54:05,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-20 16:54:05,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-20 16:54:05,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-20 16:54:05,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-20 16:54:05,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-20 16:54:05,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-20 16:54:05,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-20 16:54:05,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-20 16:54:05,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-20 16:54:05,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-20 16:54:05,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-20 16:54:05,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-20 16:54:05,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-20 16:54:05,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-20 16:54:05,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-20 16:54:05,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-20 16:54:05,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-20 16:54:05,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-20 16:54:05,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-20 16:54:05,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-20 16:54:05,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-20 16:54:05,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-20 16:54:05,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-20 16:54:05,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-20 16:54:05,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-20 16:54:05,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-20 16:54:05,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-20 16:54:05,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-20 16:54:05,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-20 16:54:05,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-20 16:54:05,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-20 16:54:05,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-20 16:54:05,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-20 16:54:05,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-20 16:54:05,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-20 16:54:05,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-20 16:54:05,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-20 16:54:05,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-20 16:54:05,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-20 16:54:05,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-20 16:54:05,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-20 16:54:05,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-20 16:54:05,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-20 16:54:05,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-20 16:54:05,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-20 16:54:05,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-20 16:54:05,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-20 16:54:05,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-20 16:54:05,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-20 16:54:05,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-20 16:54:05,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-20 16:54:05,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-20 16:54:05,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-20 16:54:05,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-20 16:54:05,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-20 16:54:05,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-20 16:54:05,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-20 16:54:05,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-20 16:54:05,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-20 16:54:05,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-20 16:54:05,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-20 16:54:05,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-20 16:54:05,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-20 16:54:05,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-20 16:54:05,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-20 16:54:05,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-20 16:54:05,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-20 16:54:05,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-20 16:54:05,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-20 16:54:05,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-20 16:54:05,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-20 16:54:05,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-20 16:54:05,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-20 16:54:05,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-20 16:54:05,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-20 16:54:05,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-20 16:54:05,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-20 16:54:05,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-20 16:54:05,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-20 16:54:05,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-20 16:54:05,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-20 16:54:05,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-20 16:54:05,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-20 16:54:05,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-20 16:54:05,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-20 16:54:05,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-20 16:54:05,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-20 16:54:05,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-20 16:54:05,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-20 16:54:05,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-20 16:54:05,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-20 16:54:05,534 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008334268559567502); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.012023172538477988); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.008334268559567502); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.005916666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-20 16:54:05,534 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-20 16:54:05,535 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=8.519953421074955E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008334268559567502); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.012023172538477988); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.008334268559567502); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.005916666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-20 16:54:05,887 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 456 ms to try 5000 different iterations.  Found a solution that moves 299 regions; Going from a computed imbalance of 8.519953421074955E-4 to a new imbalance of 3.757120790440054E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.23782800878924E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0023210354127426377); MoveCostFunction : (multiplier=7.0, imbalance=0.03322222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.23782800878924E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-20 16:54:05,888 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv413788193:83 , srv76933551:83 , srv1030991426:84 , srv105945959:84 , srv109725926:84 , srv1141216837:84 , srv1152278107:84 , srv1178625889:84 , srv1241283483:84 , srv1304157763:84 , srv1381114967:84 , srv1418381078:84 , srv1486366799:84 , srv1649791449:84 , srv1912824088:84 , srv1926810950:84 , srv1976608334:84 , srv198825882:84 , srv2023303331:84 , srv2057310639:84 , srv212306007:84 , srv217705839:84 , srv225445869:84 , srv28388002:84 , srv284056510:84 , srv424000485:84 , srv54100212:84 , srv574684638:84 , srv671950351:84 , srv745220237:84 , srv756653661:84 , srv766139592:84 , srv902992818:84 , srv92723160:84 , srv1018590555:85 , srv1030556872:85 , srv1033242031:85 , srv1049181296:85 , srv1100656900:85 , srv1119756568:85 , srv1139071086:85 , srv114230355:85 , srv1342715220:85 , srv1359079743:85 , srv1418727504:85 , srv1451658494:85 , srv1466460354:85 , srv1559312092:85 , srv1566427366:85 , srv1573830714:85 , srv1604736175:85 , srv1615486875:85 , srv1616165665:85 , srv1620046101:85 , srv1690779500:85 , srv1798060972:85 , srv183265193:85 , srv1844679285:85 , srv1848428869:85 , srv2007429464:85 , srv2047294721:85 , srv2065336459:85 , srv21169281:85 , srv2130533118:85 , srv2138536062:85 , srv225986913:85 , srv227571571:85 , srv235809209:85 , srv332312699:85 , srv356355511:85 , srv370505536:85 , srv438875110:85 , srv542656975:85 , srv56606498:85 , srv703324142:85 , srv705883907:85 , srv736994020:85 , srv795108973:85 , srv798234041:85 , srv828773578:85 , srv840550686:85 , srv844729809:85 , srv860157645:85 , srv901003933:85 , srv904591973:85 , srv929095186:85 , srv953906927:85 , srv971314618:85 , srv1143167504:86 , srv1167403821:86 , srv1174205876:86 , srv1438438151:86 , srv1583609186:86 , srv29461319:86 , srv398596228:86 , srv466565767:86 , srv486293368:86 , srv607349064:86 , srv900352575:86 , srv15594995:610 }
2023-02-20 16:54:05,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-20 16:54:05,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv736994020=81, srv901003933=93, srv1690779500=38, srv2007429464=47, srv114230355=11, srv1141216837=10, srv1912824088=43, srv840550686=89, srv1926810950=44, srv756653661=83, srv225986913=58, srv28388002=61, srv466565767=71, srv1486366799=27, srv1649791449=37, srv1018590555=0, srv795108973=86, srv953906927=98, srv212306007=53, srv424000485=69, srv1152278107=13, srv54100212=73, srv332312699=64, srv76933551=85, srv1848428869=42, srv607349064=77, srv486293368=72, srv828773578=88, srv356355511=65, srv574684638=76, srv1143167504=12, srv1466460354=26, srv15594995=29, srv29461319=63, srv1033242031=3, srv198825882=46, srv1573830714=31, srv929095186=97, srv703324142=79, srv798234041=87, srv1178625889=16, srv1359079743=20, srv109725926=6, srv235809209=60, srv1418381078=22, srv1438438151=24, srv217705839=56, srv1241283483=17, srv1030991426=2, srv1030556872=1, srv1566427366=30, srv284056510=62, srv1381114967=21, srv225445869=57, srv1583609186=32, srv1559312092=28, srv1798060972=39, srv1418727504=23, srv1451658494=25, srv705883907=80, srv2130533118=54, srv370505536=66, srv92723160=96, srv1604736175=33, srv1100656900=7, srv1139071086=9, srv2065336459=51, srv2057310639=50, srv413788193=68, srv227571571=59, srv1342715220=19, srv1615486875=34, srv904591973=95, srv1304157763=18, srv438875110=70, srv542656975=74, srv2023303331=48, srv1616165665=35, srv1167403821=14, srv1844679285=41, srv745220237=82, srv1174205876=15, srv2138536062=55, srv844729809=90, srv900352575=92, srv1049181296=4, srv2047294721=49, srv671950351=78, srv183265193=40, srv21169281=52, srv1976608334=45, srv1119756568=8, srv860157645=91, srv902992818=94, srv56606498=75, srv398596228=67, srv105945959=5, srv971314618=99, srv1620046101=36, srv766139592=84} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-20 16:54:05,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-20 16:54:05,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-20 16:54:05,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-20 16:54:05,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-20 16:54:05,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-20 16:54:05,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-20 16:54:05,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-20 16:54:05,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-20 16:54:05,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-20 16:54:05,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-20 16:54:05,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-20 16:54:05,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-20 16:54:05,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-20 16:54:05,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-20 16:54:05,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-20 16:54:05,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-20 16:54:05,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-20 16:54:05,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-20 16:54:05,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-20 16:54:05,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-20 16:54:05,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-20 16:54:05,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-20 16:54:05,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-20 16:54:05,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-20 16:54:05,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-20 16:54:05,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-20 16:54:05,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-20 16:54:05,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-20 16:54:05,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-20 16:54:05,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-20 16:54:05,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-20 16:54:05,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-20 16:54:05,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-20 16:54:05,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-20 16:54:05,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-20 16:54:05,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-20 16:54:05,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-20 16:54:05,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-20 16:54:05,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-20 16:54:05,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-20 16:54:05,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-20 16:54:05,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-20 16:54:05,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-20 16:54:05,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-20 16:54:05,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-20 16:54:05,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-20 16:54:05,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-20 16:54:05,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-20 16:54:05,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-20 16:54:05,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-20 16:54:05,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-20 16:54:05,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-20 16:54:05,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-20 16:54:05,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-20 16:54:05,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-20 16:54:05,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-20 16:54:05,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-20 16:54:05,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-20 16:54:05,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-20 16:54:05,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-20 16:54:05,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-20 16:54:05,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-20 16:54:05,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-20 16:54:05,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-20 16:54:05,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-20 16:54:05,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-20 16:54:05,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-20 16:54:05,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-20 16:54:05,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-20 16:54:05,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-20 16:54:05,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-20 16:54:05,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-20 16:54:05,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-20 16:54:05,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-20 16:54:05,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-20 16:54:05,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-20 16:54:05,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-20 16:54:05,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-20 16:54:05,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-20 16:54:05,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-20 16:54:05,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-20 16:54:05,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-20 16:54:05,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-20 16:54:05,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-20 16:54:05,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-20 16:54:05,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-20 16:54:05,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-20 16:54:05,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-20 16:54:05,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-20 16:54:05,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-20 16:54:05,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-20 16:54:05,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-20 16:54:05,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-20 16:54:05,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-20 16:54:05,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-20 16:54:05,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-20 16:54:05,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-20 16:54:05,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-20 16:54:05,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-20 16:54:05,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-20 16:54:05,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-20 16:54:06,029 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.23782800878924E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0023210354127426377); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.23782800878924E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-20 16:54:06,030 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-20 16:54:06,030 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=3.54775651560768E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.23782800878924E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0023210354127426377); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.23782800878924E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-20 16:54:06,262 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 337 ms to try 5000 different iterations.  Found a solution that moves 41 regions; Going from a computed imbalance of 3.54775651560768E-5 to a new imbalance of 7.360327432387217E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0015713484026367722); MoveCostFunction : (multiplier=7.0, imbalance=0.004555555555555556); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-20 16:54:06,262 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1018590555:83 , srv1033242031:83 , srv1167403821:83 , srv1566427366:83 , srv2065336459:83 , srv29461319:83 , srv466565767:83 , srv486293368:83 , srv798234041:83 , srv844729809:83 , srv953906927:83 , srv1030556872:84 , srv1030991426:84 , srv1049181296:84 , srv109725926:84 , srv1100656900:84 , srv1119756568:84 , srv1139071086:84 , srv1141216837:84 , srv114230355:84 , srv1143167504:84 , srv1174205876:84 , srv1178625889:84 , srv1304157763:84 , srv1342715220:84 , srv1359079743:84 , srv1418381078:84 , srv1418727504:84 , srv1438438151:84 , srv1466460354:84 , srv1486366799:84 , srv1559312092:84 , srv1573830714:84 , srv1583609186:84 , srv1604736175:84 , srv1615486875:84 , srv1616165665:84 , srv1620046101:84 , srv1649791449:84 , srv1690779500:84 , srv1798060972:84 , srv183265193:84 , srv1844679285:84 , srv1848428869:84 , srv1912824088:84 , srv1926810950:84 , srv198825882:84 , srv2007429464:84 , srv2023303331:84 , srv2047294721:84 , srv2057310639:84 , srv21169281:84 , srv2130533118:84 , srv2138536062:84 , srv225445869:84 , srv225986913:84 , srv227571571:84 , srv235809209:84 , srv284056510:84 , srv332312699:84 , srv356355511:84 , srv370505536:84 , srv398596228:84 , srv413788193:84 , srv438875110:84 , srv54100212:84 , srv542656975:84 , srv56606498:84 , srv607349064:84 , srv671950351:84 , srv703324142:84 , srv705883907:84 , srv736994020:84 , srv745220237:84 , srv756653661:84 , srv766139592:84 , srv76933551:84 , srv795108973:84 , srv828773578:84 , srv840550686:84 , srv860157645:84 , srv900352575:84 , srv901003933:84 , srv902992818:84 , srv904591973:84 , srv92723160:84 , srv929095186:84 , srv971314618:84 , srv105945959:85 , srv1152278107:85 , srv1241283483:85 , srv1381114967:85 , srv1451658494:85 , srv1976608334:85 , srv212306007:85 , srv217705839:85 , srv28388002:85 , srv424000485:85 , srv574684638:85 , srv15594995:684 }
2023-02-20 16:54:06,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-20 16:54:06,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv736994020=81, srv901003933=93, srv1690779500=38, srv2007429464=47, srv114230355=11, srv1141216837=10, srv1912824088=43, srv840550686=89, srv1926810950=44, srv756653661=83, srv225986913=58, srv28388002=61, srv466565767=71, srv1486366799=27, srv1649791449=37, srv1018590555=0, srv795108973=86, srv953906927=98, srv212306007=53, srv424000485=69, srv1152278107=13, srv54100212=73, srv332312699=64, srv76933551=85, srv1848428869=42, srv607349064=77, srv486293368=72, srv828773578=88, srv356355511=65, srv574684638=76, srv1143167504=12, srv1466460354=26, srv15594995=29, srv29461319=63, srv1033242031=3, srv198825882=46, srv1573830714=31, srv929095186=97, srv703324142=79, srv798234041=87, srv1178625889=16, srv1359079743=20, srv109725926=6, srv235809209=60, srv1418381078=22, srv1438438151=24, srv217705839=56, srv1241283483=17, srv1030991426=2, srv1030556872=1, srv1566427366=30, srv284056510=62, srv1381114967=21, srv225445869=57, srv1583609186=32, srv1559312092=28, srv1798060972=39, srv1418727504=23, srv1451658494=25, srv705883907=80, srv2130533118=54, srv370505536=66, srv92723160=96, srv1604736175=33, srv1100656900=7, srv1139071086=9, srv2065336459=51, srv2057310639=50, srv413788193=68, srv227571571=59, srv1342715220=19, srv1615486875=34, srv904591973=95, srv1304157763=18, srv438875110=70, srv542656975=74, srv2023303331=48, srv1616165665=35, srv1167403821=14, srv1844679285=41, srv745220237=82, srv1174205876=15, srv2138536062=55, srv844729809=90, srv900352575=92, srv1049181296=4, srv2047294721=49, srv671950351=78, srv183265193=40, srv21169281=52, srv1976608334=45, srv1119756568=8, srv860157645=91, srv902992818=94, srv56606498=75, srv398596228=67, srv105945959=5, srv971314618=99, srv1620046101=36, srv766139592=84} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-20 16:54:06,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-20 16:54:06,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-20 16:54:06,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-20 16:54:06,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-20 16:54:06,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-20 16:54:06,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-20 16:54:06,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-20 16:54:06,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-20 16:54:06,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-20 16:54:06,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-20 16:54:06,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-20 16:54:06,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-20 16:54:06,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-20 16:54:06,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-20 16:54:06,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-20 16:54:06,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-20 16:54:06,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-20 16:54:06,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-20 16:54:06,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-20 16:54:06,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-20 16:54:06,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-20 16:54:06,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-20 16:54:06,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-20 16:54:06,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-20 16:54:06,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-20 16:54:06,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-20 16:54:06,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-20 16:54:06,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-20 16:54:06,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-20 16:54:06,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-20 16:54:06,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-20 16:54:06,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-20 16:54:06,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-20 16:54:06,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-20 16:54:06,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-20 16:54:06,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-20 16:54:06,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-20 16:54:06,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-20 16:54:06,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-20 16:54:06,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-20 16:54:06,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-20 16:54:06,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-20 16:54:06,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-20 16:54:06,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-20 16:54:06,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-20 16:54:06,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-20 16:54:06,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-20 16:54:06,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-20 16:54:06,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-20 16:54:06,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-20 16:54:06,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-20 16:54:06,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-20 16:54:06,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-20 16:54:06,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-20 16:54:06,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-20 16:54:06,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-20 16:54:06,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-20 16:54:06,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-20 16:54:06,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-20 16:54:06,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-20 16:54:06,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-20 16:54:06,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-20 16:54:06,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-20 16:54:06,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-20 16:54:06,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-20 16:54:06,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-20 16:54:06,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-20 16:54:06,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-20 16:54:06,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-20 16:54:06,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-20 16:54:06,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-20 16:54:06,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-20 16:54:06,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-20 16:54:06,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-20 16:54:06,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-20 16:54:06,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-20 16:54:06,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-20 16:54:06,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-20 16:54:06,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-20 16:54:06,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-20 16:54:06,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-20 16:54:06,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-20 16:54:06,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-20 16:54:06,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-20 16:54:06,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-20 16:54:06,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-20 16:54:06,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-20 16:54:06,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-20 16:54:06,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-20 16:54:06,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-20 16:54:06,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-20 16:54:06,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-20 16:54:06,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-20 16:54:06,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-20 16:54:06,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-20 16:54:06,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-20 16:54:06,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-20 16:54:06,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-20 16:54:06,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-20 16:54:06,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-20 16:54:06,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-20 16:54:06,390 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=7.073239296329447E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0015713484026367722); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-20 16:54:06,391 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1018590555:83 , srv1033242031:83 , srv1167403821:83 , srv1566427366:83 , srv2065336459:83 , srv29461319:83 , srv466565767:83 , srv486293368:83 , srv798234041:83 , srv844729809:83 , srv953906927:83 , srv1030556872:84 , srv1030991426:84 , srv1049181296:84 , srv109725926:84 , srv1100656900:84 , srv1119756568:84 , srv1139071086:84 , srv1141216837:84 , srv114230355:84 , srv1143167504:84 , srv1174205876:84 , srv1178625889:84 , srv1304157763:84 , srv1342715220:84 , srv1359079743:84 , srv1418381078:84 , srv1418727504:84 , srv1438438151:84 , srv1466460354:84 , srv1486366799:84 , srv1559312092:84 , srv1573830714:84 , srv1583609186:84 , srv1604736175:84 , srv1615486875:84 , srv1616165665:84 , srv1620046101:84 , srv1649791449:84 , srv1690779500:84 , srv1798060972:84 , srv183265193:84 , srv1844679285:84 , srv1848428869:84 , srv1912824088:84 , srv1926810950:84 , srv198825882:84 , srv2007429464:84 , srv2023303331:84 , srv2047294721:84 , srv2057310639:84 , srv21169281:84 , srv2130533118:84 , srv2138536062:84 , srv225445869:84 , srv225986913:84 , srv227571571:84 , srv235809209:84 , srv284056510:84 , srv332312699:84 , srv356355511:84 , srv370505536:84 , srv398596228:84 , srv413788193:84 , srv438875110:84 , srv54100212:84 , srv542656975:84 , srv56606498:84 , srv607349064:84 , srv671950351:84 , srv703324142:84 , srv705883907:84 , srv736994020:84 , srv745220237:84 , srv756653661:84 , srv766139592:84 , srv76933551:84 , srv795108973:84 , srv828773578:84 , srv840550686:84 , srv860157645:84 , srv900352575:84 , srv901003933:84 , srv902992818:84 , srv904591973:84 , srv92723160:84 , srv929095186:84 , srv971314618:84 , srv105945959:85 , srv1152278107:85 , srv1241283483:85 , srv1381114967:85 , srv1451658494:85 , srv1976608334:85 , srv212306007:85 , srv217705839:85 , srv28388002:85 , srv424000485:85 , srv574684638:85 , srv15594995:684 }
2023-02-20 16:54:06,416 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=288 (was 295), ProcessCount=172 (was 172), AvailableMemoryMB=6043 (was 6552)
2023-02-20 16:54:06,426 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=288, ProcessCount=172, AvailableMemoryMB=6043
2023-02-20 16:54:06,426 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-20 16:54:06,427 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-02-20 16:54:06,438 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1026210920:84 , srv1060576991:84 , srv1250511079:84 , srv1256187673:84 , srv1286114868:84 , srv143687205:84 , srv1528951389:84 , srv1695050133:84 , srv1697612098:84 , srv1765057823:84 , srv1906691602:84 , srv1952826452:84 , srv2088980197:84 , srv2098378334:84 , srv210384292:84 , srv219103314:84 , srv279564531:84 , srv282716341:84 , srv319735399:84 , srv417999041:84 , srv54312739:84 , srv673537879:84 , srv673967316:84 , srv734223763:84 , srv737559209:84 , srv879495866:84 , srv892704428:84 , srv90588230:84 , srv923469066:84 , srv2041916377:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-02-20 16:54:06,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-20 16:54:06,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1250511079=2, srv1695050133=7, srv2041916377=12, srv1026210920=0, srv1060576991=1, srv417999041=20, srv1256187673=3, srv282716341=18, srv1528951389=6, srv1906691602=10, srv673537879=22, srv673967316=23, srv319735399=19, srv734223763=24, srv923469066=29, srv210384292=15, srv90588230=28, srv2098378334=14, srv2088980197=13, srv54312739=21, srv892704428=27, srv1286114868=4, srv219103314=16, srv1765057823=9, srv879495866=26, srv1697612098=8, srv143687205=5, srv1952826452=11, srv737559209=25, srv279564531=17} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-20 16:54:06,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-20 16:54:06,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-20 16:54:06,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-20 16:54:06,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-20 16:54:06,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-20 16:54:06,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-20 16:54:06,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-20 16:54:06,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-20 16:54:06,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-20 16:54:06,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-20 16:54:06,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-20 16:54:06,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-20 16:54:06,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-20 16:54:06,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-20 16:54:06,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-20 16:54:06,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-20 16:54:06,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-20 16:54:06,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-20 16:54:06,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-20 16:54:06,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-20 16:54:06,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-20 16:54:06,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-20 16:54:06,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-20 16:54:06,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-20 16:54:06,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-20 16:54:06,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-20 16:54:06,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-20 16:54:06,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-20 16:54:06,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-20 16:54:06,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-20 16:54:06,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-02-20 16:54:06,480 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0859099857504702, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-20 16:54:06,480 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912463742824161, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0859099857504702, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-02-20 16:56:06,461 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6191959 different iterations.  Found a solution that moves 1853 regions; Going from a computed imbalance of 0.9912463742824161 to a new imbalance of 4.324994439959734E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6862962962962963, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-20 16:56:06,467 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1026210920:90 , srv1060576991:90 , srv1250511079:90 , srv1256187673:90 , srv1286114868:90 , srv143687205:90 , srv1528951389:90 , srv1695050133:90 , srv1697612098:90 , srv1765057823:90 , srv1906691602:90 , srv1952826452:90 , srv2041916377:90 , srv2088980197:90 , srv2098378334:90 , srv210384292:90 , srv219103314:90 , srv279564531:90 , srv282716341:90 , srv319735399:90 , srv417999041:90 , srv54312739:90 , srv673537879:90 , srv673967316:90 , srv734223763:90 , srv737559209:90 , srv879495866:90 , srv892704428:90 , srv90588230:90 , srv923469066:90 }
2023-02-20 16:56:06,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-20 16:56:06,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1250511079=2, srv1695050133=7, srv2041916377=12, srv1026210920=0, srv1060576991=1, srv417999041=20, srv1256187673=3, srv282716341=18, srv1528951389=6, srv1906691602=10, srv673537879=22, srv673967316=23, srv319735399=19, srv734223763=24, srv923469066=29, srv210384292=15, srv90588230=28, srv2098378334=14, srv2088980197=13, srv54312739=21, srv892704428=27, srv1286114868=4, srv219103314=16, srv1765057823=9, srv879495866=26, srv1697612098=8, srv143687205=5, srv1952826452=11, srv737559209=25, srv279564531=17} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-20 16:56:06,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-20 16:56:06,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-20 16:56:06,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-20 16:56:06,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-20 16:56:06,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-20 16:56:06,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-20 16:56:06,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-20 16:56:06,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-20 16:56:06,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-20 16:56:06,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-20 16:56:06,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-20 16:56:06,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-20 16:56:06,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-20 16:56:06,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-20 16:56:06,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-20 16:56:06,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-20 16:56:06,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-20 16:56:06,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-20 16:56:06,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-20 16:56:06,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-20 16:56:06,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-20 16:56:06,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-20 16:56:06,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-20 16:56:06,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-20 16:56:06,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-20 16:56:06,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-20 16:56:06,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-20 16:56:06,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-20 16:56:06,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-20 16:56:06,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-20 16:56:06,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-02-20 16:56:06,485 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-20 16:56:06,485 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1026210920:90 , srv1060576991:90 , srv1250511079:90 , srv1256187673:90 , srv1286114868:90 , srv143687205:90 , srv1528951389:90 , srv1695050133:90 , srv1697612098:90 , srv1765057823:90 , srv1906691602:90 , srv1952826452:90 , srv2041916377:90 , srv2088980197:90 , srv2098378334:90 , srv210384292:90 , srv219103314:90 , srv279564531:90 , srv282716341:90 , srv319735399:90 , srv417999041:90 , srv54312739:90 , srv673537879:90 , srv673967316:90 , srv734223763:90 , srv737559209:90 , srv879495866:90 , srv892704428:90 , srv90588230:90 , srv923469066:90 }
2023-02-20 16:56:06,499 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=268 (was 288), ProcessCount=172 (was 172), AvailableMemoryMB=5960 (was 6043)