2023-03-21 06:56:00,471 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-03-21 06:56:00,875 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-03-21 06:56:01,028 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-03-21 06:56:01,115 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-21 06:56:01,125 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-03-21 06:56:01,145 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=310, ProcessCount=172, AvailableMemoryMB=3997
2023-03-21 06:56:01,148 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-21 06:56:01,148 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-03-21 06:56:01,322 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1014913441:84 , srv1015443417:84 , srv1015486181:84 , srv1035884888:84 , srv1038193466:84 , srv1080952625:84 , srv1093194658:84 , srv1102089690:84 , srv1108838908:84 , srv112411981:84 , srv114419871:84 , srv1196469135:84 , srv1204918949:84 , srv1206019081:84 , srv1223513076:84 , srv1226351990:84 , srv1251067280:84 , srv1265649139:84 , srv1267383601:84 , srv1375068511:84 , srv1408148362:84 , srv1432552424:84 , srv1435319507:84 , srv1438885115:84 , srv144378006:84 , srv1449141537:84 , srv1497901725:84 , srv1521826366:84 , srv1523218665:84 , srv1539795592:84 , srv1557726896:84 , srv1582592470:84 , srv1596981026:84 , srv1604471452:84 , srv1620864264:84 , srv1635450399:84 , srv1638576464:84 , srv1720590512:84 , srv1794805796:84 , srv1816975746:84 , srv1819116900:84 , srv1850143322:84 , srv1856390232:84 , srv1895365610:84 , srv1921373125:84 , srv1930569252:84 , srv1965651576:84 , srv1970537160:84 , srv199038312:84 , srv2035837831:84 , srv2036697374:84 , srv2069623258:84 , srv2083971919:84 , srv208751734:84 , srv2112058795:84 , srv2112622415:84 , srv2128809816:84 , srv2139301198:84 , srv25946081:84 , srv284312121:84 , srv298368264:84 , srv306979204:84 , srv319375355:84 , srv344334900:84 , srv364750729:84 , srv366095972:84 , srv409700273:84 , srv437758461:84 , srv499011623:84 , srv50755164:84 , srv525055228:84 , srv548475164:84 , srv552353919:84 , srv553092561:84 , srv578001823:84 , srv585176189:84 , srv620627226:84 , srv624007003:84 , srv670058989:84 , srv674493697:84 , srv683763824:84 , srv691695737:84 , srv715682802:84 , srv718811350:84 , srv733877326:84 , srv770837701:84 , srv791519708:84 , srv806048179:84 , srv811457136:84 , srv85407467:84 , srv858675274:84 , srv912564596:84 , srv917339815:84 , srv931926703:84 , srv946149446:84 , srv964424713:84 , srv987352289:84 , srv987462327:84 , srv994010387:84 , srv892966817:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-03-21 06:56:01,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-21 06:56:01,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1015486181=2, srv1265649139=17, srv1102089690=7, srv674493697=79, srv1080952625=5, srv1435319507=22, srv1539795592=29, srv25946081=58, srv670058989=78, srv1267383601=18, srv552353919=72, srv50755164=69, srv585176189=75, srv624007003=77, srv1035884888=3, srv1108838908=8, srv1638576464=36, srv1794805796=38, srv1850143322=41, srv946149446=95, srv1921373125=44, srv1596981026=32, srv208751734=53, srv858675274=90, srv553092561=73, srv1523218665=28, srv912564596=92, srv1720590512=37, srv1038193466=4, srv1557726896=30, srv964424713=96, srv683763824=80, srv806048179=87, srv2035837831=49, srv1856390232=42, srv1226351990=15, srv548475164=71, srv1204918949=12, srv298368264=60, srv2112058795=54, srv409700273=66, srv85407467=89, srv1620864264=34, srv525055228=70, srv1251067280=16, srv1408148362=20, srv1432552424=21, srv284312121=59, srv1521826366=27, srv1635450399=35, srv437758461=67, srv1375068511=19, srv733877326=84, srv691695737=81, srv1819116900=40, srv364750729=64, srv715682802=82, srv1497901725=26, srv2083971919=52, srv1895365610=43, srv1223513076=14, srv1582592470=31, srv994010387=99, srv892966817=91, srv917339815=93, srv499011623=68, srv2112622415=55, srv811457136=88, srv1093194658=6, srv1449141537=25, srv770837701=85, srv1930569252=45, srv366095972=65, srv112411981=9, srv1970537160=47, srv2069623258=51, srv1196469135=11, srv144378006=24, srv718811350=83, srv2128809816=56, srv319375355=62, srv2036697374=50, srv114419871=10, srv2139301198=57, srv306979204=61, srv344334900=63, srv1604471452=33, srv199038312=48, srv931926703=94, srv1014913441=0, srv1965651576=46, srv791519708=86, srv1206019081=13, srv987462327=98, srv578001823=74, srv620627226=76, srv1438885115=23, srv1816975746=39, srv987352289=97, srv1015443417=1} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-21 06:56:01,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-21 06:56:01,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-21 06:56:01,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-21 06:56:01,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-21 06:56:01,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-21 06:56:01,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-21 06:56:01,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-21 06:56:01,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-21 06:56:01,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-21 06:56:01,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-21 06:56:01,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-21 06:56:01,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-21 06:56:01,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-21 06:56:01,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-21 06:56:01,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-21 06:56:01,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-21 06:56:01,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-21 06:56:01,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-21 06:56:01,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-21 06:56:01,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-21 06:56:01,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-21 06:56:01,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-21 06:56:01,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-21 06:56:01,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-21 06:56:01,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-21 06:56:01,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-21 06:56:01,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-21 06:56:01,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-21 06:56:01,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-21 06:56:01,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-21 06:56:01,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-21 06:56:01,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-21 06:56:01,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-21 06:56:01,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-21 06:56:01,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-21 06:56:01,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-21 06:56:01,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-21 06:56:01,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-21 06:56:01,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-21 06:56:01,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-21 06:56:01,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-21 06:56:01,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-21 06:56:01,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-21 06:56:01,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-21 06:56:01,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-21 06:56:01,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-21 06:56:01,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-21 06:56:01,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-21 06:56:01,503 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-21 06:56:01,503 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-21 06:56:01,503 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-21 06:56:01,503 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-21 06:56:01,503 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-21 06:56:01,504 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-21 06:56:01,504 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-21 06:56:01,504 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-21 06:56:01,504 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-21 06:56:01,504 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-21 06:56:01,504 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-21 06:56:01,504 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-21 06:56:01,505 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-21 06:56:01,505 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-21 06:56:01,505 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-21 06:56:01,505 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-21 06:56:01,505 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-21 06:56:01,505 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-21 06:56:01,505 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-21 06:56:01,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-21 06:56:01,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-21 06:56:01,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-21 06:56:01,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-21 06:56:01,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-21 06:56:01,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-21 06:56:01,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-21 06:56:01,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-21 06:56:01,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-21 06:56:01,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-21 06:56:01,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-21 06:56:01,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-21 06:56:01,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-21 06:56:01,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-21 06:56:01,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-21 06:56:01,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-21 06:56:01,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-21 06:56:01,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-21 06:56:01,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-21 06:56:01,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-21 06:56:01,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-21 06:56:01,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-21 06:56:01,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-21 06:56:01,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-21 06:56:01,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-21 06:56:01,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-21 06:56:01,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-21 06:56:01,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-21 06:56:01,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-21 06:56:01,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-21 06:56:01,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-21 06:56:01,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-21 06:56:01,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-21 06:56:01,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-21 06:56:01,786 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-21 06:56:01,787 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-21 06:56:01,787 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-21 06:56:02,126 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 563 ms to try 5000 different iterations.  Found a solution that moves 4656 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18069479610508632. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04004611383543805, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05697972545876103, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5173333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04004611383543805, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17591666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24258333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-21 06:56:02,143 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1921373125:81 , srv1375068511:82 , srv364750729:82 , srv552353919:82 , srv987462327:82 , srv1015486181:83 , srv1794805796:83 , srv1930569252:83 , srv715682802:83 , srv1015443417:84 , srv1035884888:84 , srv1080952625:84 , srv1432552424:84 , srv1438885115:84 , srv1557726896:84 , srv2069623258:84 , srv2128809816:84 , srv366095972:84 , srv50755164:84 , srv674493697:84 , srv811457136:84 , srv85407467:84 , srv912564596:84 , srv114419871:85 , srv1449141537:85 , srv1620864264:85 , srv1816975746:85 , srv1850143322:85 , srv298368264:85 , srv409700273:85 , srv548475164:85 , srv553092561:85 , srv620627226:85 , srv670058989:85 , srv683763824:85 , srv733877326:85 , srv1093194658:86 , srv1204918949:86 , srv1223513076:86 , srv1435319507:86 , srv144378006:86 , srv1497901725:86 , srv1523218665:86 , srv1856390232:86 , srv2083971919:86 , srv208751734:86 , srv2112622415:86 , srv2139301198:86 , srv319375355:86 , srv624007003:86 , srv691695737:86 , srv791519708:86 , srv806048179:86 , srv987352289:86 , srv1014913441:87 , srv1102089690:87 , srv1539795592:87 , srv1720590512:87 , srv1819116900:87 , srv1970537160:87 , srv199038312:87 , srv499011623:87 , srv578001823:87 , srv585176189:87 , srv770837701:87 , srv931926703:87 , srv112411981:88 , srv1196469135:88 , srv1206019081:88 , srv1582592470:88 , srv1604471452:88 , srv1635450399:88 , srv1638576464:88 , srv1895365610:88 , srv1965651576:88 , srv25946081:88 , srv306979204:88 , srv437758461:88 , srv525055228:88 , srv718811350:88 , srv858675274:88 , srv917339815:88 , srv994010387:88 , srv1251067280:89 , srv1408148362:89 , srv2112058795:89 , srv284312121:89 , srv946149446:89 , srv964424713:89 , srv1038193466:90 , srv1108838908:90 , srv1267383601:90 , srv1596981026:90 , srv2035837831:90 , srv1226351990:91 , srv1265649139:91 , srv2036697374:91 , srv344334900:92 , srv1521826366:95 , srv892966817:446 }
2023-03-21 06:56:02,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-21 06:56:02,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1015486181=2, srv1265649139=17, srv1102089690=7, srv674493697=79, srv1080952625=5, srv1435319507=22, srv1539795592=29, srv25946081=58, srv670058989=78, srv1267383601=18, srv552353919=72, srv50755164=69, srv585176189=75, srv624007003=77, srv1035884888=3, srv1108838908=8, srv1638576464=36, srv1794805796=38, srv1850143322=41, srv946149446=95, srv1921373125=44, srv1596981026=32, srv208751734=53, srv858675274=90, srv553092561=73, srv1523218665=28, srv912564596=92, srv1720590512=37, srv1038193466=4, srv1557726896=30, srv964424713=96, srv683763824=80, srv806048179=87, srv2035837831=49, srv1856390232=42, srv1226351990=15, srv548475164=71, srv1204918949=12, srv298368264=60, srv2112058795=54, srv409700273=66, srv85407467=89, srv1620864264=34, srv525055228=70, srv1251067280=16, srv1408148362=20, srv1432552424=21, srv284312121=59, srv1521826366=27, srv1635450399=35, srv437758461=67, srv1375068511=19, srv733877326=84, srv691695737=81, srv1819116900=40, srv364750729=64, srv715682802=82, srv1497901725=26, srv2083971919=52, srv1895365610=43, srv1223513076=14, srv1582592470=31, srv994010387=99, srv892966817=91, srv917339815=93, srv499011623=68, srv2112622415=55, srv811457136=88, srv1093194658=6, srv1449141537=25, srv770837701=85, srv1930569252=45, srv366095972=65, srv112411981=9, srv1970537160=47, srv2069623258=51, srv1196469135=11, srv144378006=24, srv718811350=83, srv2128809816=56, srv319375355=62, srv2036697374=50, srv114419871=10, srv2139301198=57, srv306979204=61, srv344334900=63, srv1604471452=33, srv199038312=48, srv931926703=94, srv1014913441=0, srv1965651576=46, srv791519708=86, srv1206019081=13, srv987462327=98, srv578001823=74, srv620627226=76, srv1438885115=23, srv1816975746=39, srv987352289=97, srv1015443417=1} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-21 06:56:02,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-21 06:56:02,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-21 06:56:02,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-21 06:56:02,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-21 06:56:02,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-21 06:56:02,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-21 06:56:02,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-21 06:56:02,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-21 06:56:02,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-21 06:56:02,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-21 06:56:02,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-21 06:56:02,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-21 06:56:02,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-21 06:56:02,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-21 06:56:02,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-21 06:56:02,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-21 06:56:02,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-21 06:56:02,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-21 06:56:02,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-21 06:56:02,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-21 06:56:02,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-21 06:56:02,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-21 06:56:02,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-21 06:56:02,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-21 06:56:02,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-21 06:56:02,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-21 06:56:02,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-21 06:56:02,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-21 06:56:02,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-21 06:56:02,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-21 06:56:02,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-21 06:56:02,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-21 06:56:02,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-21 06:56:02,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-21 06:56:02,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-21 06:56:02,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-21 06:56:02,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-21 06:56:02,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-21 06:56:02,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-21 06:56:02,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-21 06:56:02,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-21 06:56:02,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-21 06:56:02,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-21 06:56:02,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-21 06:56:02,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-21 06:56:02,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-21 06:56:02,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-21 06:56:02,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-21 06:56:02,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-21 06:56:02,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-21 06:56:02,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-21 06:56:02,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-21 06:56:02,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-21 06:56:02,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-21 06:56:02,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-21 06:56:02,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-21 06:56:02,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-21 06:56:02,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-21 06:56:02,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-21 06:56:02,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-21 06:56:02,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-21 06:56:02,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-21 06:56:02,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-21 06:56:02,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-21 06:56:02,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-21 06:56:02,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-21 06:56:02,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-21 06:56:02,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-21 06:56:02,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-21 06:56:02,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-21 06:56:02,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-21 06:56:02,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-21 06:56:02,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-21 06:56:02,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-21 06:56:02,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-21 06:56:02,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-21 06:56:02,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-21 06:56:02,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-21 06:56:02,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-21 06:56:02,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-21 06:56:02,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-21 06:56:02,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-21 06:56:02,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-21 06:56:02,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-21 06:56:02,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-21 06:56:02,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-21 06:56:02,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-21 06:56:02,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-21 06:56:02,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-21 06:56:02,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-21 06:56:02,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-21 06:56:02,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-21 06:56:02,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-21 06:56:02,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-21 06:56:02,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-21 06:56:02,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-21 06:56:02,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-21 06:56:02,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-21 06:56:02,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-21 06:56:02,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-21 06:56:02,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-21 06:56:02,368 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04004611383543805, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05697972545876103, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04004611383543805, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17591666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24258333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-21 06:56:02,369 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-21 06:56:02,370 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18066219409626963, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04004611383543805, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05697972545876103, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04004611383543805, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17591666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24258333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-21 06:56:02,749 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 530 ms to try 5000 different iterations.  Found a solution that moves 2639 regions; Going from a computed imbalance of 0.18066219409626963 to a new imbalance of 0.013400849009232583. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021603623496951375); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0331645558775394, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2932222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021603623496951375); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010333333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0425, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-21 06:56:02,771 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1521826366:77 , srv2035837831:79 , srv1265649139:80 , srv112411981:82 , srv1226351990:82 , srv1251067280:82 , srv1638576464:82 , srv2036697374:82 , srv2112058795:82 , srv917339815:82 , srv964424713:82 , srv1038193466:83 , srv1267383601:83 , srv1539795592:83 , srv1596981026:83 , srv1635450399:83 , srv1720590512:83 , srv1970537160:83 , srv284312121:83 , srv344334900:83 , srv994010387:83 , srv1102089690:84 , srv1108838908:84 , srv1408148362:84 , srv1523218665:84 , srv1604471452:84 , srv1895365610:84 , srv1965651576:84 , srv437758461:84 , srv525055228:84 , srv770837701:84 , srv858675274:84 , srv946149446:84 , srv1014913441:85 , srv1093194658:85 , srv1204918949:85 , srv1206019081:85 , srv144378006:85 , srv1856390232:85 , srv199038312:85 , srv208751734:85 , srv25946081:85 , srv298368264:85 , srv553092561:85 , srv585176189:85 , srv624007003:85 , srv691695737:85 , srv718811350:85 , srv931926703:85 , srv1035884888:86 , srv114419871:86 , srv1196469135:86 , srv1223513076:86 , srv1435319507:86 , srv1497901725:86 , srv1582592470:86 , srv1819116900:86 , srv1850143322:86 , srv2112622415:86 , srv2139301198:86 , srv306979204:86 , srv319375355:86 , srv499011623:86 , srv548475164:86 , srv85407467:86 , srv987352289:86 , srv1080952625:87 , srv1438885115:87 , srv1449141537:87 , srv1557726896:87 , srv50755164:87 , srv578001823:87 , srv670058989:87 , srv674493697:87 , srv791519708:87 , srv1015443417:88 , srv1432552424:88 , srv1620864264:88 , srv620627226:88 , srv683763824:88 , srv715682802:88 , srv733877326:88 , srv806048179:88 , srv1015486181:89 , srv1930569252:89 , srv2083971919:89 , srv2128809816:89 , srv366095972:89 , srv552353919:89 , srv912564596:89 , srv1816975746:90 , srv2069623258:90 , srv409700273:90 , srv1375068511:91 , srv1921373125:91 , srv364750729:91 , srv987462327:91 , srv1794805796:92 , srv811457136:92 , srv892966817:520 }
2023-03-21 06:56:02,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-21 06:56:02,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1015486181=2, srv1265649139=17, srv1102089690=7, srv674493697=79, srv1080952625=5, srv1435319507=22, srv1539795592=29, srv25946081=58, srv670058989=78, srv1267383601=18, srv552353919=72, srv50755164=69, srv585176189=75, srv624007003=77, srv1035884888=3, srv1108838908=8, srv1638576464=36, srv1794805796=38, srv1850143322=41, srv946149446=95, srv1921373125=44, srv1596981026=32, srv208751734=53, srv858675274=90, srv553092561=73, srv1523218665=28, srv912564596=92, srv1720590512=37, srv1038193466=4, srv1557726896=30, srv964424713=96, srv683763824=80, srv806048179=87, srv2035837831=49, srv1856390232=42, srv1226351990=15, srv548475164=71, srv1204918949=12, srv298368264=60, srv2112058795=54, srv409700273=66, srv85407467=89, srv1620864264=34, srv525055228=70, srv1251067280=16, srv1408148362=20, srv1432552424=21, srv284312121=59, srv1521826366=27, srv1635450399=35, srv437758461=67, srv1375068511=19, srv733877326=84, srv691695737=81, srv1819116900=40, srv364750729=64, srv715682802=82, srv1497901725=26, srv2083971919=52, srv1895365610=43, srv1223513076=14, srv1582592470=31, srv994010387=99, srv892966817=91, srv917339815=93, srv499011623=68, srv2112622415=55, srv811457136=88, srv1093194658=6, srv1449141537=25, srv770837701=85, srv1930569252=45, srv366095972=65, srv112411981=9, srv1970537160=47, srv2069623258=51, srv1196469135=11, srv144378006=24, srv718811350=83, srv2128809816=56, srv319375355=62, srv2036697374=50, srv114419871=10, srv2139301198=57, srv306979204=61, srv344334900=63, srv1604471452=33, srv199038312=48, srv931926703=94, srv1014913441=0, srv1965651576=46, srv791519708=86, srv1206019081=13, srv987462327=98, srv578001823=74, srv620627226=76, srv1438885115=23, srv1816975746=39, srv987352289=97, srv1015443417=1} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-21 06:56:02,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-21 06:56:02,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-21 06:56:02,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-21 06:56:02,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-21 06:56:02,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-21 06:56:02,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-21 06:56:02,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-21 06:56:02,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-21 06:56:02,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-21 06:56:02,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-21 06:56:02,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-21 06:56:02,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-21 06:56:02,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-21 06:56:02,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-21 06:56:02,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-21 06:56:02,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-21 06:56:02,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-21 06:56:02,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-21 06:56:02,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-21 06:56:02,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-21 06:56:02,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-21 06:56:02,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-21 06:56:02,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-21 06:56:02,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-21 06:56:02,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-21 06:56:02,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-21 06:56:02,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-21 06:56:02,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-21 06:56:02,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-21 06:56:02,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-21 06:56:02,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-21 06:56:02,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-21 06:56:02,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-21 06:56:02,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-21 06:56:02,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-21 06:56:02,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-21 06:56:02,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-21 06:56:02,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-21 06:56:02,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-21 06:56:02,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-21 06:56:02,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-21 06:56:02,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-21 06:56:02,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-21 06:56:02,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-21 06:56:02,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-21 06:56:02,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-21 06:56:02,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-21 06:56:02,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-21 06:56:02,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-21 06:56:02,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-21 06:56:02,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-21 06:56:02,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-21 06:56:02,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-21 06:56:02,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-21 06:56:02,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-21 06:56:02,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-21 06:56:02,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-21 06:56:02,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-21 06:56:02,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-21 06:56:02,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-21 06:56:02,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-21 06:56:02,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-21 06:56:02,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-21 06:56:02,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-21 06:56:02,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-21 06:56:02,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-21 06:56:02,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-21 06:56:02,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-21 06:56:02,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-21 06:56:02,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-21 06:56:02,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-21 06:56:02,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-21 06:56:02,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-21 06:56:02,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-21 06:56:02,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-21 06:56:02,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-21 06:56:02,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-21 06:56:02,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-21 06:56:02,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-21 06:56:02,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-21 06:56:02,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-21 06:56:02,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-21 06:56:02,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-21 06:56:02,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-21 06:56:02,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-21 06:56:02,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-21 06:56:02,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-21 06:56:02,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-21 06:56:02,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-21 06:56:02,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-21 06:56:02,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-21 06:56:02,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-21 06:56:02,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-21 06:56:02,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-21 06:56:02,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-21 06:56:02,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-21 06:56:02,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-21 06:56:02,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-21 06:56:02,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-21 06:56:02,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-21 06:56:02,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-21 06:56:02,984 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021603623496951375); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0331645558775394, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021603623496951375); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010333333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0425, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-21 06:56:02,985 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-21 06:56:02,985 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.013382370336279986, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021603623496951375); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0331645558775394, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021603623496951375); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010333333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0425, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-21 06:56:03,401 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 561 ms to try 5000 different iterations.  Found a solution that moves 971 regions; Going from a computed imbalance of 0.013382370336279986 to a new imbalance of 3.0584700519281025E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0069001392512032505); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009051542148371656); MoveCostFunction : (multiplier=7.0, imbalance=0.1078888888888889, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0069001392512032505); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0025); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-21 06:56:03,404 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1816975746:82 , srv1206019081:83 , srv1267383601:83 , srv1582592470:83 , srv2069623258:83 , srv2083971919:83 , srv306979204:83 , srv344334900:83 , srv409700273:83 , srv806048179:83 , srv811457136:83 , srv912564596:83 , srv987462327:83 , srv1108838908:84 , srv1196469135:84 , srv1226351990:84 , srv1375068511:84 , srv1794805796:84 , srv2036697374:84 , srv2128809816:84 , srv25946081:84 , srv364750729:84 , srv366095972:84 , srv578001823:84 , srv718811350:84 , srv733877326:84 , srv791519708:84 , srv946149446:84 , srv1015443417:85 , srv1038193466:85 , srv1102089690:85 , srv1223513076:85 , srv1408148362:85 , srv1432552424:85 , srv1435319507:85 , srv1438885115:85 , srv1449141537:85 , srv1596981026:85 , srv1604471452:85 , srv1620864264:85 , srv1819116900:85 , srv199038312:85 , srv2112622415:85 , srv284312121:85 , srv319375355:85 , srv437758461:85 , srv499011623:85 , srv548475164:85 , srv585176189:85 , srv670058989:85 , srv683763824:85 , srv691695737:85 , srv931926703:85 , srv964424713:85 , srv1014913441:86 , srv1015486181:86 , srv1080952625:86 , srv1093194658:86 , srv114419871:86 , srv1204918949:86 , srv1251067280:86 , srv144378006:86 , srv1635450399:86 , srv1850143322:86 , srv1895365610:86 , srv1921373125:86 , srv1965651576:86 , srv208751734:86 , srv2112058795:86 , srv2139301198:86 , srv50755164:86 , srv525055228:86 , srv552353919:86 , srv620627226:86 , srv674493697:86 , srv770837701:86 , srv858675274:86 , srv987352289:86 , srv1035884888:87 , srv112411981:87 , srv1265649139:87 , srv1497901725:87 , srv1521826366:87 , srv1539795592:87 , srv1557726896:87 , srv1720590512:87 , srv1856390232:87 , srv1930569252:87 , srv298368264:87 , srv553092561:87 , srv715682802:87 , srv85407467:87 , srv994010387:87 , srv1638576464:88 , srv1970537160:88 , srv2035837831:88 , srv624007003:88 , srv1523218665:89 , srv917339815:89 , srv892966817:553 }
2023-03-21 06:56:03,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-21 06:56:03,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1015486181=2, srv1265649139=17, srv1102089690=7, srv674493697=79, srv1080952625=5, srv1435319507=22, srv1539795592=29, srv25946081=58, srv670058989=78, srv1267383601=18, srv552353919=72, srv50755164=69, srv585176189=75, srv624007003=77, srv1035884888=3, srv1108838908=8, srv1638576464=36, srv1794805796=38, srv1850143322=41, srv946149446=95, srv1921373125=44, srv1596981026=32, srv208751734=53, srv858675274=90, srv553092561=73, srv1523218665=28, srv912564596=92, srv1720590512=37, srv1038193466=4, srv1557726896=30, srv964424713=96, srv683763824=80, srv806048179=87, srv2035837831=49, srv1856390232=42, srv1226351990=15, srv548475164=71, srv1204918949=12, srv298368264=60, srv2112058795=54, srv409700273=66, srv85407467=89, srv1620864264=34, srv525055228=70, srv1251067280=16, srv1408148362=20, srv1432552424=21, srv284312121=59, srv1521826366=27, srv1635450399=35, srv437758461=67, srv1375068511=19, srv733877326=84, srv691695737=81, srv1819116900=40, srv364750729=64, srv715682802=82, srv1497901725=26, srv2083971919=52, srv1895365610=43, srv1223513076=14, srv1582592470=31, srv994010387=99, srv892966817=91, srv917339815=93, srv499011623=68, srv2112622415=55, srv811457136=88, srv1093194658=6, srv1449141537=25, srv770837701=85, srv1930569252=45, srv366095972=65, srv112411981=9, srv1970537160=47, srv2069623258=51, srv1196469135=11, srv144378006=24, srv718811350=83, srv2128809816=56, srv319375355=62, srv2036697374=50, srv114419871=10, srv2139301198=57, srv306979204=61, srv344334900=63, srv1604471452=33, srv199038312=48, srv931926703=94, srv1014913441=0, srv1965651576=46, srv791519708=86, srv1206019081=13, srv987462327=98, srv578001823=74, srv620627226=76, srv1438885115=23, srv1816975746=39, srv987352289=97, srv1015443417=1} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-21 06:56:03,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-21 06:56:03,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-21 06:56:03,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-21 06:56:03,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-21 06:56:03,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-21 06:56:03,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-21 06:56:03,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-21 06:56:03,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-21 06:56:03,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-21 06:56:03,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-21 06:56:03,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-21 06:56:03,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-21 06:56:03,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-21 06:56:03,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-21 06:56:03,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-21 06:56:03,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-21 06:56:03,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-21 06:56:03,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-21 06:56:03,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-21 06:56:03,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-21 06:56:03,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-21 06:56:03,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-21 06:56:03,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-21 06:56:03,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-21 06:56:03,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-21 06:56:03,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-21 06:56:03,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-21 06:56:03,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-21 06:56:03,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-21 06:56:03,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-21 06:56:03,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-21 06:56:03,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-21 06:56:03,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-21 06:56:03,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-21 06:56:03,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-21 06:56:03,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-21 06:56:03,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-21 06:56:03,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-21 06:56:03,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-21 06:56:03,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-21 06:56:03,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-21 06:56:03,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-21 06:56:03,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-21 06:56:03,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-21 06:56:03,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-21 06:56:03,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-21 06:56:03,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-21 06:56:03,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-21 06:56:03,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-21 06:56:03,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-21 06:56:03,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-21 06:56:03,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-21 06:56:03,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-21 06:56:03,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-21 06:56:03,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-21 06:56:03,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-21 06:56:03,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-21 06:56:03,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-21 06:56:03,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-21 06:56:03,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-21 06:56:03,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-21 06:56:03,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-21 06:56:03,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-21 06:56:03,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-21 06:56:03,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-21 06:56:03,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-21 06:56:03,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-21 06:56:03,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-21 06:56:03,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-21 06:56:03,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-21 06:56:03,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-21 06:56:03,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-21 06:56:03,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-21 06:56:03,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-21 06:56:03,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-21 06:56:03,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-21 06:56:03,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-21 06:56:03,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-21 06:56:03,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-21 06:56:03,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-21 06:56:03,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-21 06:56:03,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-21 06:56:03,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-21 06:56:03,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-21 06:56:03,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-21 06:56:03,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-21 06:56:03,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-21 06:56:03,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-21 06:56:03,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-21 06:56:03,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-21 06:56:03,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-21 06:56:03,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-21 06:56:03,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-21 06:56:03,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-21 06:56:03,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-21 06:56:03,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-21 06:56:03,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-21 06:56:03,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-21 06:56:03,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-21 06:56:03,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-21 06:56:03,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-21 06:56:03,558 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0069001392512032505); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009051542148371656); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0069001392512032505); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0025); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-21 06:56:03,558 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-21 06:56:03,559 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=2.9904791787300307E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0069001392512032505); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009051542148371656); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0069001392512032505); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0025); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-21 06:56:03,926 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 471 ms to try 5000 different iterations.  Found a solution that moves 197 regions; Going from a computed imbalance of 2.9904791787300307E-4 to a new imbalance of 1.0769709066029316E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.5313429231318235E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017082358106351274); MoveCostFunction : (multiplier=7.0, imbalance=0.02188888888888889); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.5313429231318235E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-21 06:56:03,928 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1497901725:83 , srv1521826366:83 , srv1523218665:83 , srv1794805796:83 , srv1930569252:83 , srv620627226:83 , srv624007003:83 , srv811457136:83 , srv917339815:83 , srv1014913441:84 , srv1015486181:84 , srv1038193466:84 , srv1108838908:84 , srv1196469135:84 , srv1265649139:84 , srv1375068511:84 , srv1408148362:84 , srv1432552424:84 , srv1449141537:84 , srv1557726896:84 , srv1596981026:84 , srv1620864264:84 , srv1638576464:84 , srv1819116900:84 , srv1856390232:84 , srv1895365610:84 , srv1921373125:84 , srv1965651576:84 , srv1970537160:84 , srv2083971919:84 , srv2139301198:84 , srv344334900:84 , srv409700273:84 , srv499011623:84 , srv525055228:84 , srv578001823:84 , srv858675274:84 , srv987352289:84 , srv994010387:84 , srv1015443417:85 , srv1035884888:85 , srv1080952625:85 , srv1093194658:85 , srv112411981:85 , srv114419871:85 , srv1204918949:85 , srv1223513076:85 , srv1226351990:85 , srv1251067280:85 , srv1435319507:85 , srv144378006:85 , srv1539795592:85 , srv1582592470:85 , srv1604471452:85 , srv1635450399:85 , srv1720590512:85 , srv1816975746:85 , srv1850143322:85 , srv199038312:85 , srv2035837831:85 , srv2036697374:85 , srv2069623258:85 , srv208751734:85 , srv2112058795:85 , srv2112622415:85 , srv2128809816:85 , srv25946081:85 , srv284312121:85 , srv298368264:85 , srv306979204:85 , srv319375355:85 , srv364750729:85 , srv437758461:85 , srv50755164:85 , srv548475164:85 , srv553092561:85 , srv585176189:85 , srv670058989:85 , srv674493697:85 , srv683763824:85 , srv715682802:85 , srv718811350:85 , srv733877326:85 , srv770837701:85 , srv791519708:85 , srv806048179:85 , srv85407467:85 , srv931926703:85 , srv1102089690:86 , srv1206019081:86 , srv1267383601:86 , srv1438885115:86 , srv366095972:86 , srv552353919:86 , srv691695737:86 , srv912564596:86 , srv946149446:86 , srv964424713:86 , srv987462327:86 , srv892966817:622 }
2023-03-21 06:56:03,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-21 06:56:03,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1015486181=2, srv1265649139=17, srv1102089690=7, srv674493697=79, srv1080952625=5, srv1435319507=22, srv1539795592=29, srv25946081=58, srv670058989=78, srv1267383601=18, srv552353919=72, srv50755164=69, srv585176189=75, srv624007003=77, srv1035884888=3, srv1108838908=8, srv1638576464=36, srv1794805796=38, srv1850143322=41, srv946149446=95, srv1921373125=44, srv1596981026=32, srv208751734=53, srv858675274=90, srv553092561=73, srv1523218665=28, srv912564596=92, srv1720590512=37, srv1038193466=4, srv1557726896=30, srv964424713=96, srv683763824=80, srv806048179=87, srv2035837831=49, srv1856390232=42, srv1226351990=15, srv548475164=71, srv1204918949=12, srv298368264=60, srv2112058795=54, srv409700273=66, srv85407467=89, srv1620864264=34, srv525055228=70, srv1251067280=16, srv1408148362=20, srv1432552424=21, srv284312121=59, srv1521826366=27, srv1635450399=35, srv437758461=67, srv1375068511=19, srv733877326=84, srv691695737=81, srv1819116900=40, srv364750729=64, srv715682802=82, srv1497901725=26, srv2083971919=52, srv1895365610=43, srv1223513076=14, srv1582592470=31, srv994010387=99, srv892966817=91, srv917339815=93, srv499011623=68, srv2112622415=55, srv811457136=88, srv1093194658=6, srv1449141537=25, srv770837701=85, srv1930569252=45, srv366095972=65, srv112411981=9, srv1970537160=47, srv2069623258=51, srv1196469135=11, srv144378006=24, srv718811350=83, srv2128809816=56, srv319375355=62, srv2036697374=50, srv114419871=10, srv2139301198=57, srv306979204=61, srv344334900=63, srv1604471452=33, srv199038312=48, srv931926703=94, srv1014913441=0, srv1965651576=46, srv791519708=86, srv1206019081=13, srv987462327=98, srv578001823=74, srv620627226=76, srv1438885115=23, srv1816975746=39, srv987352289=97, srv1015443417=1} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-21 06:56:03,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-21 06:56:03,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-21 06:56:03,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-21 06:56:03,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-21 06:56:03,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-21 06:56:03,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-21 06:56:03,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-21 06:56:03,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-21 06:56:03,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-21 06:56:03,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-21 06:56:03,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-21 06:56:03,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-21 06:56:03,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-21 06:56:03,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-21 06:56:03,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-21 06:56:03,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-21 06:56:03,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-21 06:56:03,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-21 06:56:03,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-21 06:56:03,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-21 06:56:03,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-21 06:56:03,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-21 06:56:03,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-21 06:56:03,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-21 06:56:03,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-21 06:56:03,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-21 06:56:03,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-21 06:56:03,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-21 06:56:03,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-21 06:56:03,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-21 06:56:03,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-21 06:56:03,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-21 06:56:03,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-21 06:56:03,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-21 06:56:03,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-21 06:56:03,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-21 06:56:03,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-21 06:56:03,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-21 06:56:03,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-21 06:56:03,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-21 06:56:03,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-21 06:56:03,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-21 06:56:03,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-21 06:56:03,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-21 06:56:03,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-21 06:56:03,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-21 06:56:03,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-21 06:56:03,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-21 06:56:03,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-21 06:56:03,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-21 06:56:03,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-21 06:56:03,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-21 06:56:03,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-21 06:56:03,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-21 06:56:03,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-21 06:56:03,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-21 06:56:03,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-21 06:56:03,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-21 06:56:03,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-21 06:56:03,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-21 06:56:03,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-21 06:56:03,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-21 06:56:03,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-21 06:56:03,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-21 06:56:03,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-21 06:56:03,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-21 06:56:03,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-21 06:56:03,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-21 06:56:03,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-21 06:56:03,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-21 06:56:03,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-21 06:56:03,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-21 06:56:03,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-21 06:56:03,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-21 06:56:03,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-21 06:56:03,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-21 06:56:03,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-21 06:56:03,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-21 06:56:03,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-21 06:56:03,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-21 06:56:03,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-21 06:56:03,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-21 06:56:03,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-21 06:56:03,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-21 06:56:03,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-21 06:56:03,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-21 06:56:03,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-21 06:56:03,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-21 06:56:03,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-21 06:56:03,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-21 06:56:03,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-21 06:56:03,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-21 06:56:03,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-21 06:56:03,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-21 06:56:03,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-21 06:56:03,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-21 06:56:03,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-21 06:56:03,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-21 06:56:03,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-21 06:56:03,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-21 06:56:03,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-21 06:56:04,105 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=9.390285583020033E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.5313429231318235E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017082358106351274); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.5313429231318235E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-21 06:56:04,105 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1497901725:83 , srv1521826366:83 , srv1523218665:83 , srv1794805796:83 , srv1930569252:83 , srv620627226:83 , srv624007003:83 , srv811457136:83 , srv917339815:83 , srv1014913441:84 , srv1015486181:84 , srv1038193466:84 , srv1108838908:84 , srv1196469135:84 , srv1265649139:84 , srv1375068511:84 , srv1408148362:84 , srv1432552424:84 , srv1449141537:84 , srv1557726896:84 , srv1596981026:84 , srv1620864264:84 , srv1638576464:84 , srv1819116900:84 , srv1856390232:84 , srv1895365610:84 , srv1921373125:84 , srv1965651576:84 , srv1970537160:84 , srv2083971919:84 , srv2139301198:84 , srv344334900:84 , srv409700273:84 , srv499011623:84 , srv525055228:84 , srv578001823:84 , srv858675274:84 , srv987352289:84 , srv994010387:84 , srv1015443417:85 , srv1035884888:85 , srv1080952625:85 , srv1093194658:85 , srv112411981:85 , srv114419871:85 , srv1204918949:85 , srv1223513076:85 , srv1226351990:85 , srv1251067280:85 , srv1435319507:85 , srv144378006:85 , srv1539795592:85 , srv1582592470:85 , srv1604471452:85 , srv1635450399:85 , srv1720590512:85 , srv1816975746:85 , srv1850143322:85 , srv199038312:85 , srv2035837831:85 , srv2036697374:85 , srv2069623258:85 , srv208751734:85 , srv2112058795:85 , srv2112622415:85 , srv2128809816:85 , srv25946081:85 , srv284312121:85 , srv298368264:85 , srv306979204:85 , srv319375355:85 , srv364750729:85 , srv437758461:85 , srv50755164:85 , srv548475164:85 , srv553092561:85 , srv585176189:85 , srv670058989:85 , srv674493697:85 , srv683763824:85 , srv715682802:85 , srv718811350:85 , srv733877326:85 , srv770837701:85 , srv791519708:85 , srv806048179:85 , srv85407467:85 , srv931926703:85 , srv1102089690:86 , srv1206019081:86 , srv1267383601:86 , srv1438885115:86 , srv366095972:86 , srv552353919:86 , srv691695737:86 , srv912564596:86 , srv946149446:86 , srv964424713:86 , srv987462327:86 , srv892966817:622 }
2023-03-21 06:56:04,146 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=310 (was 310), ProcessCount=170 (was 172), AvailableMemoryMB=3511 (was 3997)
2023-03-21 06:56:04,163 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=310, ProcessCount=170, AvailableMemoryMB=3511
2023-03-21 06:56:04,164 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-21 06:56:04,164 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-03-21 06:56:04,184 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1050803171:84 , srv111714173:84 , srv1176769721:84 , srv1193320155:84 , srv1295588079:84 , srv1349573162:84 , srv1499167947:84 , srv1568411703:84 , srv1574873816:84 , srv160041520:84 , srv1633283031:84 , srv1653060978:84 , srv1677710504:84 , srv1808226270:84 , srv1823100240:84 , srv1843644645:84 , srv1886823581:84 , srv1918332401:84 , srv1959950998:84 , srv1966055611:84 , srv250885439:84 , srv28708513:84 , srv300775491:84 , srv468498830:84 , srv602103169:84 , srv622701233:84 , srv626304468:84 , srv904953315:84 , srv986880276:84 , srv1962363372:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-03-21 06:56:04,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-21 06:56:04,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1843644645=15, srv1677710504=12, srv111714173=1, srv1962363372=19, srv986880276=29, srv1295588079=4, srv1966055611=20, srv300775491=23, srv622701233=26, srv1808226270=13, srv1886823581=16, srv1959950998=18, srv250885439=21, srv1193320155=3, srv28708513=22, srv602103169=25, srv1176769721=2, srv1633283031=10, srv160041520=9, srv1823100240=14, srv1499167947=6, srv1574873816=8, srv1349573162=5, srv1918332401=17, srv904953315=28, srv468498830=24, srv1050803171=0, srv1568411703=7, srv1653060978=11, srv626304468=27} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-21 06:56:04,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-21 06:56:04,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-21 06:56:04,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-21 06:56:04,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-21 06:56:04,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-21 06:56:04,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-21 06:56:04,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-21 06:56:04,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-21 06:56:04,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-21 06:56:04,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-21 06:56:04,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-21 06:56:04,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-21 06:56:04,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-21 06:56:04,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-21 06:56:04,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-21 06:56:04,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-21 06:56:04,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-21 06:56:04,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-21 06:56:04,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-21 06:56:04,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-21 06:56:04,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-21 06:56:04,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-21 06:56:04,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-21 06:56:04,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-21 06:56:04,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-21 06:56:04,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-21 06:56:04,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-21 06:56:04,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-21 06:56:04,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-21 06:56:04,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-21 06:56:04,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-03-21 06:56:04,221 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1006464817358839, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-21 06:56:04,221 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.991251017704182, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1006464817358839, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-03-21 06:58:04,204 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5704392 different iterations.  Found a solution that moves 1852 regions; Going from a computed imbalance of 0.991251017704182 to a new imbalance of 4.322660390073086E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6859259259259259, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-21 06:58:04,210 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1050803171:90 , srv111714173:90 , srv1176769721:90 , srv1193320155:90 , srv1295588079:90 , srv1349573162:90 , srv1499167947:90 , srv1568411703:90 , srv1574873816:90 , srv160041520:90 , srv1633283031:90 , srv1653060978:90 , srv1677710504:90 , srv1808226270:90 , srv1823100240:90 , srv1843644645:90 , srv1886823581:90 , srv1918332401:90 , srv1959950998:90 , srv1962363372:90 , srv1966055611:90 , srv250885439:90 , srv28708513:90 , srv300775491:90 , srv468498830:90 , srv602103169:90 , srv622701233:90 , srv626304468:90 , srv904953315:90 , srv986880276:90 }
2023-03-21 06:58:04,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-21 06:58:04,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1843644645=15, srv1677710504=12, srv111714173=1, srv1962363372=19, srv986880276=29, srv1295588079=4, srv1966055611=20, srv300775491=23, srv622701233=26, srv1808226270=13, srv1886823581=16, srv1959950998=18, srv250885439=21, srv1193320155=3, srv28708513=22, srv602103169=25, srv1176769721=2, srv1633283031=10, srv160041520=9, srv1823100240=14, srv1499167947=6, srv1574873816=8, srv1349573162=5, srv1918332401=17, srv904953315=28, srv468498830=24, srv1050803171=0, srv1568411703=7, srv1653060978=11, srv626304468=27} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-21 06:58:04,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-21 06:58:04,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-21 06:58:04,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-21 06:58:04,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-21 06:58:04,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-21 06:58:04,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-21 06:58:04,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-21 06:58:04,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-21 06:58:04,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-21 06:58:04,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-21 06:58:04,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-21 06:58:04,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-21 06:58:04,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-21 06:58:04,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-21 06:58:04,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-21 06:58:04,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-21 06:58:04,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-21 06:58:04,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-21 06:58:04,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-21 06:58:04,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-21 06:58:04,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-21 06:58:04,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-21 06:58:04,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-21 06:58:04,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-21 06:58:04,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-21 06:58:04,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-21 06:58:04,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-21 06:58:04,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-21 06:58:04,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-21 06:58:04,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-21 06:58:04,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-03-21 06:58:04,228 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-21 06:58:04,228 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1050803171:90 , srv111714173:90 , srv1176769721:90 , srv1193320155:90 , srv1295588079:90 , srv1349573162:90 , srv1499167947:90 , srv1568411703:90 , srv1574873816:90 , srv160041520:90 , srv1633283031:90 , srv1653060978:90 , srv1677710504:90 , srv1808226270:90 , srv1823100240:90 , srv1843644645:90 , srv1886823581:90 , srv1918332401:90 , srv1959950998:90 , srv1962363372:90 , srv1966055611:90 , srv250885439:90 , srv28708513:90 , srv300775491:90 , srv468498830:90 , srv602103169:90 , srv622701233:90 , srv626304468:90 , srv904953315:90 , srv986880276:90 }
2023-03-21 06:58:04,240 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=265 (was 310), ProcessCount=170 (was 170), AvailableMemoryMB=3636 (was 3511) - AvailableMemoryMB LEAK? -