2023-02-13 17:56:11,450 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-02-13 17:56:11,780 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-02-13 17:56:11,891 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-02-13 17:56:11,955 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-13 17:56:11,963 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-02-13 17:56:11,976 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=122, ProcessCount=168, AvailableMemoryMB=1669
2023-02-13 17:56:11,978 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-13 17:56:11,979 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-02-13 17:56:12,089 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1053628840:84 , srv1055316650:84 , srv1068123640:84 , srv1125396889:84 , srv113930130:84 , srv1171277286:84 , srv1178049967:84 , srv1179201334:84 , srv1209000380:84 , srv1250360158:84 , srv1274666768:84 , srv1281101825:84 , srv1286050492:84 , srv1297279518:84 , srv1310435557:84 , srv1317745998:84 , srv1335122111:84 , srv1360834159:84 , srv1363208016:84 , srv1364838270:84 , srv1378583812:84 , srv1417792152:84 , srv144187354:84 , srv1443516446:84 , srv1447988100:84 , srv1490078531:84 , srv1502156827:84 , srv1514132564:84 , srv1532558378:84 , srv1537816693:84 , srv1558028917:84 , srv1562205765:84 , srv1594425620:84 , srv1635864947:84 , srv1646310096:84 , srv1649993357:84 , srv1661876538:84 , srv1673760524:84 , srv1692374567:84 , srv1704507721:84 , srv1712796322:84 , srv1749697413:84 , srv1786227152:84 , srv179315908:84 , srv1846081698:84 , srv1857924073:84 , srv187195376:84 , srv1879245529:84 , srv1881813958:84 , srv1888669614:84 , srv1896684205:84 , srv193942532:84 , srv1963262762:84 , srv1967913905:84 , srv1968444367:84 , srv1970034884:84 , srv1979823026:84 , srv2090313786:84 , srv211267491:84 , srv213263473:84 , srv2140811474:84 , srv215822436:84 , srv232938695:84 , srv243033925:84 , srv253301632:84 , srv254788708:84 , srv284163502:84 , srv314619654:84 , srv325969010:84 , srv330913027:84 , srv332351849:84 , srv342395140:84 , srv348241835:84 , srv357654341:84 , srv381981537:84 , srv393164335:84 , srv400186930:84 , srv410870088:84 , srv427970030:84 , srv450327365:84 , srv483915312:84 , srv507120480:84 , srv54540062:84 , srv552281226:84 , srv613566872:84 , srv644457492:84 , srv669644076:84 , srv705632414:84 , srv792860320:84 , srv807723819:84 , srv809526302:84 , srv818711551:84 , srv826117512:84 , srv888816037:84 , srv892906822:84 , srv930474158:84 , srv939234195:84 , srv98239482:84 , srv98944537:84 , srv175121661:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-02-13 17:56:12,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-13 17:56:12,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv892906822=95, srv1417792152=21, srv1635864947=33, srv330913027=70, srv1447988100=24, srv1963262762=53, srv1712796322=40, srv1562205765=31, srv792860320=89, srv1649993357=35, srv1297279518=13, srv1646310096=34, srv427970030=79, srv348241835=73, srv325969010=69, srv410870088=78, srv284163502=67, srv1209000380=8, srv1363208016=18, srv1786227152=43, srv232938695=63, srv818711551=92, srv939234195=97, srv1692374567=38, srv809526302=91, srv930474158=96, srv243033925=64, srv253301632=65, srv314619654=68, srv807723819=90, srv888816037=94, srv1594425620=32, srv113930130=4, srv507120480=82, srv1274666768=10, srv1250360158=9, srv215822436=62, srv211267491=59, srv144187354=22, srv1537816693=29, srv98239482=98, srv1514132564=27, srv1888669614=50, srv644457492=86, srv826117512=93, srv254788708=66, srv1317745998=15, srv1053628840=0, srv669644076=87, srv1125396889=3, srv1970034884=56, srv332351849=71, srv1335122111=16, srv1178049967=6, srv1490078531=25, srv342395140=72, srv1310435557=14, srv179315908=44, srv1881813958=49, srv1378583812=20, srv1673760524=37, srv613566872=85, srv381981537=75, srv1661876538=36, srv1968444367=55, srv1281101825=11, srv175121661=42, srv2090313786=58, srv1749697413=41, srv98944537=99, srv1979823026=57, srv705632414=88, srv1846081698=45, srv1364838270=19, srv393164335=76, srv400186930=77, srv483915312=81, srv1879245529=48, srv2140811474=61, srv1179201334=7, srv213263473=60, srv1055316650=1, srv1068123640=2, srv1967913905=54, srv1532558378=28, srv1704507721=39, srv187195376=47, srv552281226=84, srv1443516446=23, srv1360834159=17, srv193942532=52, srv1171277286=5, srv450327365=80, srv1502156827=26, srv357654341=74, srv1857924073=46, srv1896684205=51, srv1286050492=12, srv54540062=83, srv1558028917=30} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-13 17:56:12,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-13 17:56:12,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-13 17:56:12,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-13 17:56:12,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-13 17:56:12,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-13 17:56:12,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-13 17:56:12,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-13 17:56:12,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-13 17:56:12,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-13 17:56:12,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-13 17:56:12,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-13 17:56:12,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-13 17:56:12,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-13 17:56:12,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-13 17:56:12,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-13 17:56:12,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-13 17:56:12,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-13 17:56:12,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-13 17:56:12,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-13 17:56:12,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-13 17:56:12,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-13 17:56:12,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-13 17:56:12,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-13 17:56:12,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-13 17:56:12,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-13 17:56:12,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-13 17:56:12,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-13 17:56:12,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-13 17:56:12,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-13 17:56:12,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-13 17:56:12,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-13 17:56:12,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-13 17:56:12,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-13 17:56:12,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-13 17:56:12,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-13 17:56:12,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-13 17:56:12,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-13 17:56:12,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-13 17:56:12,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-13 17:56:12,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-13 17:56:12,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-13 17:56:12,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-13 17:56:12,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-13 17:56:12,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-13 17:56:12,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-13 17:56:12,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-13 17:56:12,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-13 17:56:12,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-13 17:56:12,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-13 17:56:12,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-13 17:56:12,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-13 17:56:12,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-13 17:56:12,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-13 17:56:12,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-13 17:56:12,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-13 17:56:12,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-13 17:56:12,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-13 17:56:12,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-13 17:56:12,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-13 17:56:12,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-13 17:56:12,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-13 17:56:12,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-13 17:56:12,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-13 17:56:12,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-13 17:56:12,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-13 17:56:12,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-13 17:56:12,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-13 17:56:12,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-13 17:56:12,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-13 17:56:12,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-13 17:56:12,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-13 17:56:12,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-13 17:56:12,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-13 17:56:12,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-13 17:56:12,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-13 17:56:12,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-13 17:56:12,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-13 17:56:12,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-13 17:56:12,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-13 17:56:12,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-13 17:56:12,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-13 17:56:12,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-13 17:56:12,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-13 17:56:12,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-13 17:56:12,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-13 17:56:12,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-13 17:56:12,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-13 17:56:12,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-13 17:56:12,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-13 17:56:12,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-13 17:56:12,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-13 17:56:12,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-13 17:56:12,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-13 17:56:12,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-13 17:56:12,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-13 17:56:12,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-13 17:56:12,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-13 17:56:12,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-13 17:56:12,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-13 17:56:12,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-13 17:56:12,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-13 17:56:12,351 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-13 17:56:12,351 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-13 17:56:12,351 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-13 17:56:12,621 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 402 ms to try 5000 different iterations.  Found a solution that moves 4721 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1723840445553156. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04233784157854373, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.056493099208106155, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5245555555555556, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04233784157854373, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.16716666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23766666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-13 17:56:12,631 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1360834159:80 , srv1661876538:80 , srv809526302:80 , srv1970034884:81 , srv213263473:81 , srv507120480:81 , srv113930130:82 , srv1179201334:82 , srv1712796322:82 , srv357654341:82 , srv54540062:82 , srv1310435557:83 , srv1378583812:83 , srv1562205765:83 , srv1594425620:83 , srv1673760524:83 , srv179315908:83 , srv193942532:83 , srv232938695:83 , srv253301632:83 , srv1171277286:84 , srv1209000380:84 , srv1281101825:84 , srv1490078531:84 , srv1846081698:84 , srv1879245529:84 , srv254788708:84 , srv348241835:84 , srv705632414:84 , srv892906822:84 , srv1250360158:85 , srv1286050492:85 , srv1417792152:85 , srv1514132564:85 , srv1704507721:85 , srv1857924073:85 , srv1979823026:85 , srv2090313786:85 , srv243033925:85 , srv381981537:85 , srv483915312:85 , srv1053628840:86 , srv1274666768:86 , srv1635864947:86 , srv1786227152:86 , srv1963262762:86 , srv215822436:86 , srv325969010:86 , srv330913027:86 , srv342395140:86 , srv427970030:86 , srv644457492:86 , srv1125396889:87 , srv1364838270:87 , srv1443516446:87 , srv1447988100:87 , srv1502156827:87 , srv1532558378:87 , srv1692374567:87 , srv1881813958:87 , srv1888669614:87 , srv1968444367:87 , srv2140811474:87 , srv284163502:87 , srv314619654:87 , srv332351849:87 , srv613566872:87 , srv669644076:87 , srv807723819:87 , srv818711551:87 , srv1178049967:88 , srv1537816693:88 , srv1749697413:88 , srv888816037:88 , srv930474158:88 , srv98944537:88 , srv1068123640:89 , srv1363208016:89 , srv144187354:89 , srv1967913905:89 , srv211267491:89 , srv552281226:89 , srv1055316650:90 , srv1297279518:90 , srv1317745998:90 , srv1335122111:90 , srv1646310096:90 , srv187195376:90 , srv393164335:90 , srv410870088:90 , srv792860320:90 , srv826117512:90 , srv939234195:90 , srv1558028917:91 , srv1896684205:91 , srv450327365:91 , srv400186930:92 , srv1649993357:95 , srv98239482:95 , srv175121661:466 }
2023-02-13 17:56:12,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-13 17:56:12,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv892906822=95, srv1417792152=21, srv1635864947=33, srv330913027=70, srv1447988100=24, srv1963262762=53, srv1712796322=40, srv1562205765=31, srv792860320=89, srv1649993357=35, srv1297279518=13, srv1646310096=34, srv427970030=79, srv348241835=73, srv325969010=69, srv410870088=78, srv284163502=67, srv1209000380=8, srv1363208016=18, srv1786227152=43, srv232938695=63, srv818711551=92, srv939234195=97, srv1692374567=38, srv809526302=91, srv930474158=96, srv243033925=64, srv253301632=65, srv314619654=68, srv807723819=90, srv888816037=94, srv1594425620=32, srv113930130=4, srv507120480=82, srv1274666768=10, srv1250360158=9, srv215822436=62, srv211267491=59, srv144187354=22, srv1537816693=29, srv98239482=98, srv1514132564=27, srv1888669614=50, srv644457492=86, srv826117512=93, srv254788708=66, srv1317745998=15, srv1053628840=0, srv669644076=87, srv1125396889=3, srv1970034884=56, srv332351849=71, srv1335122111=16, srv1178049967=6, srv1490078531=25, srv342395140=72, srv1310435557=14, srv179315908=44, srv1881813958=49, srv1378583812=20, srv1673760524=37, srv613566872=85, srv381981537=75, srv1661876538=36, srv1968444367=55, srv1281101825=11, srv175121661=42, srv2090313786=58, srv1749697413=41, srv98944537=99, srv1979823026=57, srv705632414=88, srv1846081698=45, srv1364838270=19, srv393164335=76, srv400186930=77, srv483915312=81, srv1879245529=48, srv2140811474=61, srv1179201334=7, srv213263473=60, srv1055316650=1, srv1068123640=2, srv1967913905=54, srv1532558378=28, srv1704507721=39, srv187195376=47, srv552281226=84, srv1443516446=23, srv1360834159=17, srv193942532=52, srv1171277286=5, srv450327365=80, srv1502156827=26, srv357654341=74, srv1857924073=46, srv1896684205=51, srv1286050492=12, srv54540062=83, srv1558028917=30} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-13 17:56:12,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-13 17:56:12,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-13 17:56:12,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-13 17:56:12,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-13 17:56:12,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-13 17:56:12,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-13 17:56:12,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-13 17:56:12,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-13 17:56:12,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-13 17:56:12,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-13 17:56:12,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-13 17:56:12,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-13 17:56:12,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-13 17:56:12,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-13 17:56:12,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-13 17:56:12,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-13 17:56:12,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-13 17:56:12,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-13 17:56:12,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-13 17:56:12,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-13 17:56:12,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-13 17:56:12,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-13 17:56:12,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-13 17:56:12,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-13 17:56:12,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-13 17:56:12,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-13 17:56:12,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-13 17:56:12,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-13 17:56:12,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-13 17:56:12,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-13 17:56:12,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-13 17:56:12,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-13 17:56:12,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-13 17:56:12,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-13 17:56:12,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-13 17:56:12,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-13 17:56:12,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-13 17:56:12,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-13 17:56:12,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-13 17:56:12,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-13 17:56:12,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-13 17:56:12,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-13 17:56:12,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-13 17:56:12,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-13 17:56:12,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-13 17:56:12,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-13 17:56:12,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-13 17:56:12,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-13 17:56:12,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-13 17:56:12,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-13 17:56:12,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-13 17:56:12,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-13 17:56:12,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-13 17:56:12,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-13 17:56:12,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-13 17:56:12,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-13 17:56:12,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-13 17:56:12,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-13 17:56:12,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-13 17:56:12,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-13 17:56:12,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-13 17:56:12,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-13 17:56:12,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-13 17:56:12,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-13 17:56:12,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-13 17:56:12,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-13 17:56:12,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-13 17:56:12,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-13 17:56:12,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-13 17:56:12,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-13 17:56:12,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-13 17:56:12,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-13 17:56:12,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-13 17:56:12,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-13 17:56:12,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-13 17:56:12,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-13 17:56:12,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-13 17:56:12,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-13 17:56:12,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-13 17:56:12,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-13 17:56:12,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-13 17:56:12,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-13 17:56:12,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-13 17:56:12,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-13 17:56:12,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-13 17:56:12,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-13 17:56:12,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-13 17:56:12,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-13 17:56:12,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-13 17:56:12,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-13 17:56:12,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-13 17:56:12,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-13 17:56:12,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-13 17:56:12,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-13 17:56:12,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-13 17:56:12,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-13 17:56:12,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-13 17:56:12,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-13 17:56:12,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-13 17:56:12,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-13 17:56:12,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-13 17:56:12,770 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04233784157854373, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.056493099208106155, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04233784157854373, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.16716666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23766666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-13 17:56:12,770 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-13 17:56:12,770 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17235098740677104, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04233784157854373, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.056493099208106155, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04233784157854373, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.16716666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23766666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-13 17:56:13,079 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 411 ms to try 5000 different iterations.  Found a solution that moves 2640 regions; Going from a computed imbalance of 0.17235098740677104 to a new imbalance of 0.014774628359202456. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022057752762120332); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0395343263881271, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29333333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022057752762120332); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01125); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04825, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-13 17:56:13,101 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1649993357:76 , srv98239482:78 , srv1558028917:80 , srv187195376:81 , srv1896684205:81 , srv400186930:81 , srv792860320:81 , srv393164335:82 , srv826117512:82 , srv888816037:82 , srv939234195:82 , srv1178049967:83 , srv1317745998:83 , srv144187354:83 , srv1502156827:83 , srv211267491:83 , srv450327365:83 , srv613566872:83 , srv669644076:83 , srv930474158:83 , srv1068123640:84 , srv1297279518:84 , srv1335122111:84 , srv1363208016:84 , srv1532558378:84 , srv1692374567:84 , srv1749697413:84 , srv1888669614:84 , srv2140811474:84 , srv332351849:84 , srv342395140:84 , srv552281226:84 , srv1055316650:85 , srv1125396889:85 , srv1443516446:85 , srv1646310096:85 , srv1704507721:85 , srv1968444367:85 , srv243033925:85 , srv284163502:85 , srv314619654:85 , srv818711551:85 , srv98944537:85 , srv1274666768:86 , srv1286050492:86 , srv1447988100:86 , srv1537816693:86 , srv1786227152:86 , srv1857924073:86 , srv1967913905:86 , srv215822436:86 , srv254788708:86 , srv325969010:86 , srv348241835:86 , srv483915312:86 , srv705632414:86 , srv807723819:86 , srv1053628840:87 , srv1171277286:87 , srv1209000380:87 , srv1250360158:87 , srv1310435557:87 , srv1364838270:87 , srv1378583812:87 , srv1417792152:87 , srv1514132564:87 , srv1635864947:87 , srv1881813958:87 , srv1963262762:87 , srv2090313786:87 , srv253301632:87 , srv330913027:87 , srv381981537:87 , srv410870088:87 , srv1490078531:88 , srv1562205765:88 , srv1594425620:88 , srv1673760524:88 , srv179315908:88 , srv1846081698:88 , srv1979823026:88 , srv54540062:88 , srv892906822:88 , srv1970034884:89 , srv232938695:89 , srv357654341:89 , srv644457492:89 , srv1179201334:90 , srv1281101825:90 , srv1712796322:90 , srv1879245529:90 , srv193942532:90 , srv213263473:90 , srv427970030:90 , srv113930130:91 , srv1661876538:91 , srv507120480:91 , srv809526302:92 , srv1360834159:94 , srv175121661:504 }
2023-02-13 17:56:13,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-13 17:56:13,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv892906822=95, srv1417792152=21, srv1635864947=33, srv330913027=70, srv1447988100=24, srv1963262762=53, srv1712796322=40, srv1562205765=31, srv792860320=89, srv1649993357=35, srv1297279518=13, srv1646310096=34, srv427970030=79, srv348241835=73, srv325969010=69, srv410870088=78, srv284163502=67, srv1209000380=8, srv1363208016=18, srv1786227152=43, srv232938695=63, srv818711551=92, srv939234195=97, srv1692374567=38, srv809526302=91, srv930474158=96, srv243033925=64, srv253301632=65, srv314619654=68, srv807723819=90, srv888816037=94, srv1594425620=32, srv113930130=4, srv507120480=82, srv1274666768=10, srv1250360158=9, srv215822436=62, srv211267491=59, srv144187354=22, srv1537816693=29, srv98239482=98, srv1514132564=27, srv1888669614=50, srv644457492=86, srv826117512=93, srv254788708=66, srv1317745998=15, srv1053628840=0, srv669644076=87, srv1125396889=3, srv1970034884=56, srv332351849=71, srv1335122111=16, srv1178049967=6, srv1490078531=25, srv342395140=72, srv1310435557=14, srv179315908=44, srv1881813958=49, srv1378583812=20, srv1673760524=37, srv613566872=85, srv381981537=75, srv1661876538=36, srv1968444367=55, srv1281101825=11, srv175121661=42, srv2090313786=58, srv1749697413=41, srv98944537=99, srv1979823026=57, srv705632414=88, srv1846081698=45, srv1364838270=19, srv393164335=76, srv400186930=77, srv483915312=81, srv1879245529=48, srv2140811474=61, srv1179201334=7, srv213263473=60, srv1055316650=1, srv1068123640=2, srv1967913905=54, srv1532558378=28, srv1704507721=39, srv187195376=47, srv552281226=84, srv1443516446=23, srv1360834159=17, srv193942532=52, srv1171277286=5, srv450327365=80, srv1502156827=26, srv357654341=74, srv1857924073=46, srv1896684205=51, srv1286050492=12, srv54540062=83, srv1558028917=30} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-13 17:56:13,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-13 17:56:13,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-13 17:56:13,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-13 17:56:13,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-13 17:56:13,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-13 17:56:13,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-13 17:56:13,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-13 17:56:13,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-13 17:56:13,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-13 17:56:13,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-13 17:56:13,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-13 17:56:13,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-13 17:56:13,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-13 17:56:13,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-13 17:56:13,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-13 17:56:13,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-13 17:56:13,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-13 17:56:13,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-13 17:56:13,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-13 17:56:13,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-13 17:56:13,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-13 17:56:13,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-13 17:56:13,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-13 17:56:13,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-13 17:56:13,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-13 17:56:13,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-13 17:56:13,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-13 17:56:13,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-13 17:56:13,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-13 17:56:13,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-13 17:56:13,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-13 17:56:13,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-13 17:56:13,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-13 17:56:13,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-13 17:56:13,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-13 17:56:13,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-13 17:56:13,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-13 17:56:13,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-13 17:56:13,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-13 17:56:13,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-13 17:56:13,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-13 17:56:13,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-13 17:56:13,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-13 17:56:13,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-13 17:56:13,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-13 17:56:13,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-13 17:56:13,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-13 17:56:13,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-13 17:56:13,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-13 17:56:13,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-13 17:56:13,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-13 17:56:13,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-13 17:56:13,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-13 17:56:13,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-13 17:56:13,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-13 17:56:13,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-13 17:56:13,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-13 17:56:13,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-13 17:56:13,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-13 17:56:13,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-13 17:56:13,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-13 17:56:13,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-13 17:56:13,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-13 17:56:13,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-13 17:56:13,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-13 17:56:13,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-13 17:56:13,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-13 17:56:13,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-13 17:56:13,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-13 17:56:13,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-13 17:56:13,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-13 17:56:13,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-13 17:56:13,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-13 17:56:13,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-13 17:56:13,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-13 17:56:13,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-13 17:56:13,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-13 17:56:13,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-13 17:56:13,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-13 17:56:13,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-13 17:56:13,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-13 17:56:13,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-13 17:56:13,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-13 17:56:13,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-13 17:56:13,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-13 17:56:13,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-13 17:56:13,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-13 17:56:13,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-13 17:56:13,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-13 17:56:13,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-13 17:56:13,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-13 17:56:13,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-13 17:56:13,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-13 17:56:13,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-13 17:56:13,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-13 17:56:13,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-13 17:56:13,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-13 17:56:13,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-13 17:56:13,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-13 17:56:13,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-13 17:56:13,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-13 17:56:13,281 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022057752762120332); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0395343263881271, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022057752762120332); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01125); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04825, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-13 17:56:13,281 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-13 17:56:13,281 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.0147561426841002, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022057752762120332); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0395343263881271, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022057752762120332); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01125); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04825, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-13 17:56:13,526 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 359 ms to try 5000 different iterations.  Found a solution that moves 1060 regions; Going from a computed imbalance of 0.0147561426841002 to a new imbalance of 9.044578054225116E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006790837632067591); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.012013834150631415); MoveCostFunction : (multiplier=7.0, imbalance=0.11777777777777777, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006790837632067591); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004833333333333334); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-13 17:56:13,531 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1879245529:80 , srv410870088:81 , srv644457492:81 , srv1364838270:82 , srv427970030:82 , srv1297279518:83 , srv1363208016:83 , srv1646310096:83 , srv1881813958:83 , srv1055316650:84 , srv1068123640:84 , srv1281101825:84 , srv1335122111:84 , srv1360834159:84 , srv1537816693:84 , srv193942532:84 , srv1963262762:84 , srv1967913905:84 , srv215822436:84 , srv284163502:84 , srv330913027:84 , srv809526302:84 , srv98239482:84 , srv1053628840:85 , srv1125396889:85 , srv113930130:85 , srv1179201334:85 , srv1209000380:85 , srv1250360158:85 , srv1317745998:85 , srv144187354:85 , srv1443516446:85 , srv1447988100:85 , srv1490078531:85 , srv1635864947:85 , srv1786227152:85 , srv1896684205:85 , srv1968444367:85 , srv1979823026:85 , srv2090313786:85 , srv232938695:85 , srv381981537:85 , srv393164335:85 , srv450327365:85 , srv483915312:85 , srv552281226:85 , srv807723819:85 , srv939234195:85 , srv98944537:85 , srv1274666768:86 , srv1286050492:86 , srv1417792152:86 , srv1514132564:86 , srv1532558378:86 , srv1649993357:86 , srv1661876538:86 , srv1673760524:86 , srv1712796322:86 , srv1749697413:86 , srv1846081698:86 , srv1857924073:86 , srv187195376:86 , srv211267491:86 , srv213263473:86 , srv314619654:86 , srv325969010:86 , srv332351849:86 , srv357654341:86 , srv400186930:86 , srv507120480:86 , srv792860320:86 , srv818711551:86 , srv826117512:86 , srv930474158:86 , srv1171277286:87 , srv1178049967:87 , srv1378583812:87 , srv1502156827:87 , srv1558028917:87 , srv1562205765:87 , srv1594425620:87 , srv1692374567:87 , srv1704507721:87 , srv179315908:87 , srv1888669614:87 , srv1970034884:87 , srv2140811474:87 , srv243033925:87 , srv253301632:87 , srv254788708:87 , srv342395140:87 , srv54540062:87 , srv613566872:87 , srv669644076:87 , srv888816037:87 , srv892906822:87 , srv1310435557:88 , srv348241835:88 , srv705632414:88 , srv175121661:548 }
2023-02-13 17:56:13,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-13 17:56:13,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv892906822=95, srv1417792152=21, srv1635864947=33, srv330913027=70, srv1447988100=24, srv1963262762=53, srv1712796322=40, srv1562205765=31, srv792860320=89, srv1649993357=35, srv1297279518=13, srv1646310096=34, srv427970030=79, srv348241835=73, srv325969010=69, srv410870088=78, srv284163502=67, srv1209000380=8, srv1363208016=18, srv1786227152=43, srv232938695=63, srv818711551=92, srv939234195=97, srv1692374567=38, srv809526302=91, srv930474158=96, srv243033925=64, srv253301632=65, srv314619654=68, srv807723819=90, srv888816037=94, srv1594425620=32, srv113930130=4, srv507120480=82, srv1274666768=10, srv1250360158=9, srv215822436=62, srv211267491=59, srv144187354=22, srv1537816693=29, srv98239482=98, srv1514132564=27, srv1888669614=50, srv644457492=86, srv826117512=93, srv254788708=66, srv1317745998=15, srv1053628840=0, srv669644076=87, srv1125396889=3, srv1970034884=56, srv332351849=71, srv1335122111=16, srv1178049967=6, srv1490078531=25, srv342395140=72, srv1310435557=14, srv179315908=44, srv1881813958=49, srv1378583812=20, srv1673760524=37, srv613566872=85, srv381981537=75, srv1661876538=36, srv1968444367=55, srv1281101825=11, srv175121661=42, srv2090313786=58, srv1749697413=41, srv98944537=99, srv1979823026=57, srv705632414=88, srv1846081698=45, srv1364838270=19, srv393164335=76, srv400186930=77, srv483915312=81, srv1879245529=48, srv2140811474=61, srv1179201334=7, srv213263473=60, srv1055316650=1, srv1068123640=2, srv1967913905=54, srv1532558378=28, srv1704507721=39, srv187195376=47, srv552281226=84, srv1443516446=23, srv1360834159=17, srv193942532=52, srv1171277286=5, srv450327365=80, srv1502156827=26, srv357654341=74, srv1857924073=46, srv1896684205=51, srv1286050492=12, srv54540062=83, srv1558028917=30} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-13 17:56:13,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-13 17:56:13,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-13 17:56:13,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-13 17:56:13,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-13 17:56:13,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-13 17:56:13,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-13 17:56:13,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-13 17:56:13,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-13 17:56:13,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-13 17:56:13,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-13 17:56:13,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-13 17:56:13,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-13 17:56:13,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-13 17:56:13,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-13 17:56:13,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-13 17:56:13,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-13 17:56:13,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-13 17:56:13,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-13 17:56:13,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-13 17:56:13,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-13 17:56:13,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-13 17:56:13,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-13 17:56:13,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-13 17:56:13,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-13 17:56:13,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-13 17:56:13,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-13 17:56:13,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-13 17:56:13,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-13 17:56:13,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-13 17:56:13,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-13 17:56:13,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-13 17:56:13,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-13 17:56:13,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-13 17:56:13,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-13 17:56:13,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-13 17:56:13,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-13 17:56:13,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-13 17:56:13,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-13 17:56:13,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-13 17:56:13,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-13 17:56:13,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-13 17:56:13,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-13 17:56:13,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-13 17:56:13,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-13 17:56:13,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-13 17:56:13,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-13 17:56:13,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-13 17:56:13,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-13 17:56:13,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-13 17:56:13,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-13 17:56:13,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-13 17:56:13,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-13 17:56:13,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-13 17:56:13,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-13 17:56:13,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-13 17:56:13,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-13 17:56:13,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-13 17:56:13,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-13 17:56:13,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-13 17:56:13,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-13 17:56:13,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-13 17:56:13,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-13 17:56:13,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-13 17:56:13,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-13 17:56:13,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-13 17:56:13,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-13 17:56:13,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-13 17:56:13,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-13 17:56:13,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-13 17:56:13,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-13 17:56:13,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-13 17:56:13,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-13 17:56:13,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-13 17:56:13,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-13 17:56:13,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-13 17:56:13,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-13 17:56:13,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-13 17:56:13,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-13 17:56:13,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-13 17:56:13,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-13 17:56:13,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-13 17:56:13,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-13 17:56:13,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-13 17:56:13,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-13 17:56:13,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-13 17:56:13,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-13 17:56:13,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-13 17:56:13,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-13 17:56:13,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-13 17:56:13,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-13 17:56:13,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-13 17:56:13,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-13 17:56:13,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-13 17:56:13,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-13 17:56:13,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-13 17:56:13,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-13 17:56:13,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-13 17:56:13,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-13 17:56:13,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-13 17:56:13,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-13 17:56:13,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-13 17:56:13,660 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006790837632067591); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.012013834150631415); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006790837632067591); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004833333333333334); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-13 17:56:13,660 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-13 17:56:13,660 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=8.970355267829693E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006790837632067591); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.012013834150631415); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006790837632067591); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004833333333333334); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-13 17:56:13,993 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 429 ms to try 5000 different iterations.  Found a solution that moves 310 regions; Going from a computed imbalance of 8.970355267829693E-4 to a new imbalance of 1.3369241263101992E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0019534423844256277); MoveCostFunction : (multiplier=7.0, imbalance=0.034444444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-13 17:56:13,994 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1967913905:82 , srv1053628840:83 , srv1055316650:83 , srv1274666768:83 , srv1635864947:83 , srv1692374567:83 , srv450327365:83 , srv892906822:83 , srv113930130:84 , srv1178049967:84 , srv1281101825:84 , srv1310435557:84 , srv1317745998:84 , srv1335122111:84 , srv1417792152:84 , srv144187354:84 , srv1443516446:84 , srv1514132564:84 , srv1537816693:84 , srv1558028917:84 , srv1562205765:84 , srv1594425620:84 , srv1646310096:84 , srv1749697413:84 , srv179315908:84 , srv1846081698:84 , srv1857924073:84 , srv1888669614:84 , srv1979823026:84 , srv211267491:84 , srv2140811474:84 , srv314619654:84 , srv348241835:84 , srv357654341:84 , srv400186930:84 , srv410870088:84 , srv427970030:84 , srv507120480:84 , srv552281226:84 , srv705632414:84 , srv807723819:84 , srv818711551:84 , srv98944537:84 , srv1068123640:85 , srv1125396889:85 , srv1171277286:85 , srv1179201334:85 , srv1250360158:85 , srv1286050492:85 , srv1297279518:85 , srv1360834159:85 , srv1378583812:85 , srv1447988100:85 , srv1490078531:85 , srv1502156827:85 , srv1532558378:85 , srv1649993357:85 , srv1661876538:85 , srv1673760524:85 , srv1704507721:85 , srv1712796322:85 , srv1786227152:85 , srv187195376:85 , srv1881813958:85 , srv1896684205:85 , srv1963262762:85 , srv1968444367:85 , srv1970034884:85 , srv2090313786:85 , srv213263473:85 , srv232938695:85 , srv243033925:85 , srv253301632:85 , srv254788708:85 , srv325969010:85 , srv330913027:85 , srv332351849:85 , srv342395140:85 , srv381981537:85 , srv393164335:85 , srv483915312:85 , srv54540062:85 , srv792860320:85 , srv826117512:85 , srv888816037:85 , srv930474158:85 , srv939234195:85 , srv98239482:85 , srv1209000380:86 , srv1363208016:86 , srv1364838270:86 , srv193942532:86 , srv215822436:86 , srv284163502:86 , srv613566872:86 , srv644457492:86 , srv669644076:86 , srv809526302:86 , srv1879245529:88 , srv175121661:624 }
2023-02-13 17:56:13,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-13 17:56:13,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv892906822=95, srv1417792152=21, srv1635864947=33, srv330913027=70, srv1447988100=24, srv1963262762=53, srv1712796322=40, srv1562205765=31, srv792860320=89, srv1649993357=35, srv1297279518=13, srv1646310096=34, srv427970030=79, srv348241835=73, srv325969010=69, srv410870088=78, srv284163502=67, srv1209000380=8, srv1363208016=18, srv1786227152=43, srv232938695=63, srv818711551=92, srv939234195=97, srv1692374567=38, srv809526302=91, srv930474158=96, srv243033925=64, srv253301632=65, srv314619654=68, srv807723819=90, srv888816037=94, srv1594425620=32, srv113930130=4, srv507120480=82, srv1274666768=10, srv1250360158=9, srv215822436=62, srv211267491=59, srv144187354=22, srv1537816693=29, srv98239482=98, srv1514132564=27, srv1888669614=50, srv644457492=86, srv826117512=93, srv254788708=66, srv1317745998=15, srv1053628840=0, srv669644076=87, srv1125396889=3, srv1970034884=56, srv332351849=71, srv1335122111=16, srv1178049967=6, srv1490078531=25, srv342395140=72, srv1310435557=14, srv179315908=44, srv1881813958=49, srv1378583812=20, srv1673760524=37, srv613566872=85, srv381981537=75, srv1661876538=36, srv1968444367=55, srv1281101825=11, srv175121661=42, srv2090313786=58, srv1749697413=41, srv98944537=99, srv1979823026=57, srv705632414=88, srv1846081698=45, srv1364838270=19, srv393164335=76, srv400186930=77, srv483915312=81, srv1879245529=48, srv2140811474=61, srv1179201334=7, srv213263473=60, srv1055316650=1, srv1068123640=2, srv1967913905=54, srv1532558378=28, srv1704507721=39, srv187195376=47, srv552281226=84, srv1443516446=23, srv1360834159=17, srv193942532=52, srv1171277286=5, srv450327365=80, srv1502156827=26, srv357654341=74, srv1857924073=46, srv1896684205=51, srv1286050492=12, srv54540062=83, srv1558028917=30} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-13 17:56:14,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-13 17:56:14,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-13 17:56:14,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-13 17:56:14,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-13 17:56:14,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-13 17:56:14,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-13 17:56:14,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-13 17:56:14,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-13 17:56:14,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-13 17:56:14,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-13 17:56:14,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-13 17:56:14,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-13 17:56:14,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-13 17:56:14,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-13 17:56:14,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-13 17:56:14,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-13 17:56:14,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-13 17:56:14,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-13 17:56:14,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-13 17:56:14,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-13 17:56:14,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-13 17:56:14,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-13 17:56:14,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-13 17:56:14,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-13 17:56:14,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-13 17:56:14,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-13 17:56:14,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-13 17:56:14,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-13 17:56:14,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-13 17:56:14,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-13 17:56:14,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-13 17:56:14,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-13 17:56:14,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-13 17:56:14,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-13 17:56:14,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-13 17:56:14,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-13 17:56:14,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-13 17:56:14,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-13 17:56:14,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-13 17:56:14,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-13 17:56:14,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-13 17:56:14,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-13 17:56:14,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-13 17:56:14,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-13 17:56:14,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-13 17:56:14,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-13 17:56:14,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-13 17:56:14,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-13 17:56:14,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-13 17:56:14,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-13 17:56:14,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-13 17:56:14,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-13 17:56:14,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-13 17:56:14,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-13 17:56:14,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-13 17:56:14,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-13 17:56:14,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-13 17:56:14,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-13 17:56:14,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-13 17:56:14,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-13 17:56:14,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-13 17:56:14,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-13 17:56:14,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-13 17:56:14,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-13 17:56:14,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-13 17:56:14,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-13 17:56:14,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-13 17:56:14,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-13 17:56:14,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-13 17:56:14,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-13 17:56:14,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-13 17:56:14,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-13 17:56:14,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-13 17:56:14,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-13 17:56:14,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-13 17:56:14,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-13 17:56:14,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-13 17:56:14,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-13 17:56:14,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-13 17:56:14,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-13 17:56:14,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-13 17:56:14,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-13 17:56:14,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-13 17:56:14,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-13 17:56:14,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-13 17:56:14,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-13 17:56:14,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-13 17:56:14,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-13 17:56:14,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-13 17:56:14,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-13 17:56:14,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-13 17:56:14,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-13 17:56:14,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-13 17:56:14,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-13 17:56:14,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-13 17:56:14,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-13 17:56:14,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-13 17:56:14,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-13 17:56:14,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-13 17:56:14,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-13 17:56:14,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-13 17:56:14,126 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.1198574868518857E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0019534423844256277); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-13 17:56:14,126 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1967913905:82 , srv1053628840:83 , srv1055316650:83 , srv1274666768:83 , srv1635864947:83 , srv1692374567:83 , srv450327365:83 , srv892906822:83 , srv113930130:84 , srv1178049967:84 , srv1281101825:84 , srv1310435557:84 , srv1317745998:84 , srv1335122111:84 , srv1417792152:84 , srv144187354:84 , srv1443516446:84 , srv1514132564:84 , srv1537816693:84 , srv1558028917:84 , srv1562205765:84 , srv1594425620:84 , srv1646310096:84 , srv1749697413:84 , srv179315908:84 , srv1846081698:84 , srv1857924073:84 , srv1888669614:84 , srv1979823026:84 , srv211267491:84 , srv2140811474:84 , srv314619654:84 , srv348241835:84 , srv357654341:84 , srv400186930:84 , srv410870088:84 , srv427970030:84 , srv507120480:84 , srv552281226:84 , srv705632414:84 , srv807723819:84 , srv818711551:84 , srv98944537:84 , srv1068123640:85 , srv1125396889:85 , srv1171277286:85 , srv1179201334:85 , srv1250360158:85 , srv1286050492:85 , srv1297279518:85 , srv1360834159:85 , srv1378583812:85 , srv1447988100:85 , srv1490078531:85 , srv1502156827:85 , srv1532558378:85 , srv1649993357:85 , srv1661876538:85 , srv1673760524:85 , srv1704507721:85 , srv1712796322:85 , srv1786227152:85 , srv187195376:85 , srv1881813958:85 , srv1896684205:85 , srv1963262762:85 , srv1968444367:85 , srv1970034884:85 , srv2090313786:85 , srv213263473:85 , srv232938695:85 , srv243033925:85 , srv253301632:85 , srv254788708:85 , srv325969010:85 , srv330913027:85 , srv332351849:85 , srv342395140:85 , srv381981537:85 , srv393164335:85 , srv483915312:85 , srv54540062:85 , srv792860320:85 , srv826117512:85 , srv888816037:85 , srv930474158:85 , srv939234195:85 , srv98239482:85 , srv1209000380:86 , srv1363208016:86 , srv1364838270:86 , srv193942532:86 , srv215822436:86 , srv284163502:86 , srv613566872:86 , srv644457492:86 , srv669644076:86 , srv809526302:86 , srv1879245529:88 , srv175121661:624 }
2023-02-13 17:56:14,150 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=122 (was 122), ProcessCount=168 (was 168), AvailableMemoryMB=1189 (was 1669)
2023-02-13 17:56:14,157 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=122, ProcessCount=168, AvailableMemoryMB=1189
2023-02-13 17:56:14,158 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-13 17:56:14,158 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-02-13 17:56:14,166 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1035784818:84 , srv1060701919:84 , srv108606237:84 , srv1122111101:84 , srv1217563092:84 , srv1228744231:84 , srv1234895349:84 , srv1297411491:84 , srv1297507411:84 , srv1350204175:84 , srv1381679355:84 , srv1476507200:84 , srv1532251723:84 , srv1544654965:84 , srv1564098443:84 , srv1579172521:84 , srv1607013734:84 , srv1636039581:84 , srv1693630713:84 , srv1953773428:84 , srv211891860:84 , srv2137098412:84 , srv325835194:84 , srv389604662:84 , srv599287010:84 , srv655971825:84 , srv79711557:84 , srv884587103:84 , srv974545157:84 , srv1254943882:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-02-13 17:56:14,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-13 17:56:14,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv884587103=28, srv1297411491=8, srv1636039581=18, srv211891860=21, srv2137098412=22, srv1035784818=0, srv1254943882=7, srv1476507200=12, srv1564098443=15, srv1297507411=9, srv974545157=29, srv1544654965=14, srv389604662=24, srv1579172521=16, srv655971825=26, srv1381679355=11, srv1228744231=5, srv1234895349=6, srv1350204175=10, srv1953773428=20, srv1532251723=13, srv1607013734=17, srv325835194=23, srv108606237=2, srv599287010=25, srv1217563092=4, srv1693630713=19, srv79711557=27, srv1122111101=3, srv1060701919=1} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-13 17:56:14,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-13 17:56:14,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-13 17:56:14,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-13 17:56:14,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-13 17:56:14,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-13 17:56:14,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-13 17:56:14,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-13 17:56:14,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-13 17:56:14,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-13 17:56:14,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-13 17:56:14,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-13 17:56:14,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-13 17:56:14,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-13 17:56:14,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-13 17:56:14,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-13 17:56:14,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-13 17:56:14,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-13 17:56:14,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-13 17:56:14,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-13 17:56:14,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-13 17:56:14,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-13 17:56:14,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-13 17:56:14,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-13 17:56:14,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-13 17:56:14,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-13 17:56:14,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-13 17:56:14,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-13 17:56:14,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-13 17:56:14,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-13 17:56:14,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-13 17:56:14,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-02-13 17:56:14,193 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0754948819918873, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-13 17:56:14,193 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912430925172302, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0754948819918873, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-02-13 17:58:14,179 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6193836 different iterations.  Found a solution that moves 1854 regions; Going from a computed imbalance of 0.9912430925172302 to a new imbalance of 4.3273284898463826E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6866666666666666, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-13 17:58:14,186 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1035784818:90 , srv1060701919:90 , srv108606237:90 , srv1122111101:90 , srv1217563092:90 , srv1228744231:90 , srv1234895349:90 , srv1254943882:90 , srv1297411491:90 , srv1297507411:90 , srv1350204175:90 , srv1381679355:90 , srv1476507200:90 , srv1532251723:90 , srv1544654965:90 , srv1564098443:90 , srv1579172521:90 , srv1607013734:90 , srv1636039581:90 , srv1693630713:90 , srv1953773428:90 , srv211891860:90 , srv2137098412:90 , srv325835194:90 , srv389604662:90 , srv599287010:90 , srv655971825:90 , srv79711557:90 , srv884587103:90 , srv974545157:90 }
2023-02-13 17:58:14,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-13 17:58:14,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv884587103=28, srv1297411491=8, srv1636039581=18, srv211891860=21, srv2137098412=22, srv1035784818=0, srv1254943882=7, srv1476507200=12, srv1564098443=15, srv1297507411=9, srv974545157=29, srv1544654965=14, srv389604662=24, srv1579172521=16, srv655971825=26, srv1381679355=11, srv1228744231=5, srv1234895349=6, srv1350204175=10, srv1953773428=20, srv1532251723=13, srv1607013734=17, srv325835194=23, srv108606237=2, srv599287010=25, srv1217563092=4, srv1693630713=19, srv79711557=27, srv1122111101=3, srv1060701919=1} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-13 17:58:14,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-13 17:58:14,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-13 17:58:14,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-13 17:58:14,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-13 17:58:14,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-13 17:58:14,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-13 17:58:14,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-13 17:58:14,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-13 17:58:14,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-13 17:58:14,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-13 17:58:14,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-13 17:58:14,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-13 17:58:14,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-13 17:58:14,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-13 17:58:14,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-13 17:58:14,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-13 17:58:14,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-13 17:58:14,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-13 17:58:14,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-13 17:58:14,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-13 17:58:14,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-13 17:58:14,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-13 17:58:14,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-13 17:58:14,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-13 17:58:14,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-13 17:58:14,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-13 17:58:14,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-13 17:58:14,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-13 17:58:14,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-13 17:58:14,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-13 17:58:14,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-02-13 17:58:14,212 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-13 17:58:14,212 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1035784818:90 , srv1060701919:90 , srv108606237:90 , srv1122111101:90 , srv1217563092:90 , srv1228744231:90 , srv1234895349:90 , srv1254943882:90 , srv1297411491:90 , srv1297507411:90 , srv1350204175:90 , srv1381679355:90 , srv1476507200:90 , srv1532251723:90 , srv1544654965:90 , srv1564098443:90 , srv1579172521:90 , srv1607013734:90 , srv1636039581:90 , srv1693630713:90 , srv1953773428:90 , srv211891860:90 , srv2137098412:90 , srv325835194:90 , srv389604662:90 , srv599287010:90 , srv655971825:90 , srv79711557:90 , srv884587103:90 , srv974545157:90 }
2023-02-13 17:58:14,227 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=162 (was 122) - SystemLoadAverage LEAK? -, ProcessCount=167 (was 168), AvailableMemoryMB=1448 (was 1189) - AvailableMemoryMB LEAK? -