2023-10-17 13:53:24,477 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-10-17 13:53:24,799 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-10-17 13:53:24,903 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-10-17 13:53:24,963 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-10-17 13:53:24,972 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-10-17 13:53:24,990 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=304, ProcessCount=169, AvailableMemoryMB=8168
2023-10-17 13:53:24,993 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-10-17 13:53:24,994 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-10-17 13:53:25,162 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1001492600:84 , srv100344502:84 , srv1010735243:84 , srv1043443795:84 , srv1049333031:84 , srv1092870270:84 , srv1099533769:84 , srv1165373178:84 , srv1181199171:84 , srv118276342:84 , srv1200563167:84 , srv1287536769:84 , srv1299514362:84 , srv1342760164:84 , srv135065636:84 , srv1370707343:84 , srv1382806506:84 , srv1398917727:84 , srv1407347639:84 , srv143242153:84 , srv1455809945:84 , srv148464192:84 , srv1488921574:84 , srv1494559835:84 , srv1522876137:84 , srv1528669297:84 , srv1542458070:84 , srv1595555996:84 , srv1614094334:84 , srv1639131352:84 , srv1647583933:84 , srv1674185934:84 , srv1767070965:84 , srv1770440135:84 , srv1774339935:84 , srv1785246743:84 , srv181501488:84 , srv1833889580:84 , srv1836953328:84 , srv1838135305:84 , srv1845849854:84 , srv1874064991:84 , srv1917468271:84 , srv1927080041:84 , srv1965043102:84 , srv1975685238:84 , srv2006685054:84 , srv2026262529:84 , srv2034199265:84 , srv2059944144:84 , srv2070985487:84 , srv207102330:84 , srv2078613391:84 , srv2081621070:84 , srv2099120681:84 , srv2102186448:84 , srv232264678:84 , srv260383061:84 , srv265003565:84 , srv27603860:84 , srv280389588:84 , srv304118065:84 , srv345058517:84 , srv348459046:84 , srv368030586:84 , srv418710095:84 , srv427759035:84 , srv434850845:84 , srv435391891:84 , srv447135110:84 , srv460246314:84 , srv460815577:84 , srv491700988:84 , srv501675314:84 , srv502747216:84 , srv520172517:84 , srv522004608:84 , srv528342269:84 , srv534271845:84 , srv542778990:84 , srv546713662:84 , srv631278452:84 , srv672211332:84 , srv686321517:84 , srv695750920:84 , srv73738423:84 , srv771578911:84 , srv778908639:84 , srv78555481:84 , srv802860922:84 , srv825111372:84 , srv827258793:84 , srv832948629:84 , srv845801361:84 , srv884540661:84 , srv890286880:84 , srv928364369:84 , srv986949618:84 , srv991092790:84 , srv610030727:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-10-17 13:53:25,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-17 13:53:25,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1043443795=3, srv1595555996=27, srv148464192=21, srv1614094334=28, srv610030727=81, srv1370707343=15, srv73738423=86, srv1181199171=8, srv631278452=82, srv672211332=83, srv845801361=94, srv1165373178=7, srv520172517=75, srv1639131352=29, srv1494559835=23, srv447135110=69, srv1287536769=11, srv832948629=93, srv522004608=76, srv1833889580=37, srv345058517=62, srv528342269=77, srv1099533769=6, srv1542458070=26, srv207102330=51, srv460815577=71, srv890286880=96, srv118276342=9, srv825111372=91, srv1522876137=24, srv1770440135=33, srv1382806506=16, srv1975685238=45, srv1767070965=32, srv1845849854=40, srv778908639=88, srv434850845=67, srv1874064991=41, srv686321517=84, srv1927080041=43, srv27603860=59, srv1836953328=38, srv460246314=70, srv1001492600=0, srv1774339935=34, srv928364369=97, srv1838135305=39, srv368030586=64, srv304118065=61, srv232264678=56, srv2078613391=52, srv1965043102=44, srv491700988=72, srv1455809945=20, srv2081621070=53, srv827258793=92, srv1407347639=18, srv1092870270=5, srv1342760164=13, srv1785246743=35, srv418710095=65, srv348459046=63, srv2070985487=50, srv427759035=66, srv542778990=79, srv265003565=58, srv280389588=60, srv502747216=74, srv2006685054=46, srv884540661=95, srv1010735243=2, srv260383061=57, srv1200563167=10, srv2102186448=55, srv695750920=85, srv1647583933=30, srv2026262529=47, srv78555481=89, srv143242153=19, srv1049333031=4, srv546713662=80, srv100344502=1, srv181501488=36, srv435391891=68, srv771578911=87, srv991092790=99, srv1398917727=17, srv534271845=78, srv1917468271=42, srv2059944144=49, srv501675314=73, srv2034199265=48, srv2099120681=54, srv135065636=14, srv1488921574=22, srv802860922=90, srv1528669297=25, srv1674185934=31, srv986949618=98, srv1299514362=12} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-17 13:53:25,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-17 13:53:25,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-17 13:53:25,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-17 13:53:25,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-17 13:53:25,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-17 13:53:25,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-17 13:53:25,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-17 13:53:25,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-17 13:53:25,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-17 13:53:25,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-17 13:53:25,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-17 13:53:25,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-17 13:53:25,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-17 13:53:25,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-17 13:53:25,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-17 13:53:25,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-17 13:53:25,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-17 13:53:25,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-17 13:53:25,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-17 13:53:25,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-17 13:53:25,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-17 13:53:25,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-17 13:53:25,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-17 13:53:25,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-17 13:53:25,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-17 13:53:25,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-17 13:53:25,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-17 13:53:25,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-17 13:53:25,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-17 13:53:25,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-17 13:53:25,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-17 13:53:25,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-17 13:53:25,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-17 13:53:25,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-17 13:53:25,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-17 13:53:25,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-17 13:53:25,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-17 13:53:25,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-17 13:53:25,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-17 13:53:25,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-17 13:53:25,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-17 13:53:25,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-17 13:53:25,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-17 13:53:25,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-17 13:53:25,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-17 13:53:25,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-17 13:53:25,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-17 13:53:25,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-17 13:53:25,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-17 13:53:25,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-17 13:53:25,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-17 13:53:25,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-17 13:53:25,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-17 13:53:25,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-17 13:53:25,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-17 13:53:25,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-17 13:53:25,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-17 13:53:25,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-17 13:53:25,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-17 13:53:25,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-17 13:53:25,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-17 13:53:25,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-17 13:53:25,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-17 13:53:25,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-17 13:53:25,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-17 13:53:25,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-17 13:53:25,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-17 13:53:25,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-17 13:53:25,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-17 13:53:25,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-17 13:53:25,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-17 13:53:25,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-17 13:53:25,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-17 13:53:25,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-17 13:53:25,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-17 13:53:25,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-17 13:53:25,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-17 13:53:25,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-17 13:53:25,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-17 13:53:25,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-17 13:53:25,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-17 13:53:25,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-17 13:53:25,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-17 13:53:25,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-17 13:53:25,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-17 13:53:25,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-17 13:53:25,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-17 13:53:25,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-17 13:53:25,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-17 13:53:25,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-17 13:53:25,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-17 13:53:25,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-17 13:53:25,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-17 13:53:25,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-17 13:53:25,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-17 13:53:25,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-17 13:53:25,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-17 13:53:25,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-17 13:53:25,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-17 13:53:25,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-17 13:53:25,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-17 13:53:25,549 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-17 13:53:25,550 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-17 13:53:25,551 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-17 13:53:25,938 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 627 ms to try 5000 different iterations.  Found a solution that moves 4672 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18300902492616. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.041044171546729395, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05333543766892307, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5191111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.041044171546729395, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17816666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24591666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-17 13:53:25,951 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1099533769:82 , srv1299514362:82 , srv825111372:82 , srv1674185934:83 , srv1845849854:83 , srv2006685054:83 , srv2026262529:83 , srv232264678:83 , srv1181199171:84 , srv135065636:84 , srv1488921574:84 , srv1595555996:84 , srv1833889580:84 , srv1975685238:84 , srv2078613391:84 , srv2102186448:84 , srv520172517:84 , srv542778990:84 , srv672211332:84 , srv695750920:84 , srv832948629:84 , srv991092790:84 , srv1049333031:85 , srv1092870270:85 , srv118276342:85 , srv148464192:85 , srv1639131352:85 , srv1917468271:85 , srv207102330:85 , srv27603860:85 , srv280389588:85 , srv631278452:85 , srv771578911:85 , srv78555481:85 , srv884540661:85 , srv928364369:85 , srv1200563167:86 , srv1407347639:86 , srv143242153:86 , srv1455809945:86 , srv1542458070:86 , srv1647583933:86 , srv1767070965:86 , srv1770440135:86 , srv1838135305:86 , srv1927080041:86 , srv260383061:86 , srv265003565:86 , srv427759035:86 , srv434850845:86 , srv435391891:86 , srv460246314:86 , srv460815577:86 , srv501675314:86 , srv534271845:86 , srv546713662:86 , srv686321517:86 , srv802860922:86 , srv890286880:86 , srv1001492600:87 , srv100344502:87 , srv1398917727:87 , srv1614094334:87 , srv1774339935:87 , srv1874064991:87 , srv1965043102:87 , srv2034199265:87 , srv2081621070:87 , srv368030586:87 , srv502747216:87 , srv73738423:87 , srv778908639:87 , srv827258793:87 , srv1043443795:88 , srv1165373178:88 , srv1287536769:88 , srv1370707343:88 , srv1382806506:88 , srv1528669297:88 , srv1785246743:88 , srv181501488:88 , srv1836953328:88 , srv304118065:88 , srv418710095:88 , srv522004608:88 , srv528342269:88 , srv1342760164:89 , srv2099120681:89 , srv345058517:89 , srv1010735243:90 , srv1522876137:90 , srv447135110:90 , srv491700988:90 , srv1494559835:91 , srv2070985487:91 , srv348459046:91 , srv986949618:92 , srv2059944144:93 , srv845801361:93 , srv610030727:455 }
2023-10-17 13:53:25,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-17 13:53:25,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1043443795=3, srv1595555996=27, srv148464192=21, srv1614094334=28, srv610030727=81, srv1370707343=15, srv73738423=86, srv1181199171=8, srv631278452=82, srv672211332=83, srv845801361=94, srv1165373178=7, srv520172517=75, srv1639131352=29, srv1494559835=23, srv447135110=69, srv1287536769=11, srv832948629=93, srv522004608=76, srv1833889580=37, srv345058517=62, srv528342269=77, srv1099533769=6, srv1542458070=26, srv207102330=51, srv460815577=71, srv890286880=96, srv118276342=9, srv825111372=91, srv1522876137=24, srv1770440135=33, srv1382806506=16, srv1975685238=45, srv1767070965=32, srv1845849854=40, srv778908639=88, srv434850845=67, srv1874064991=41, srv686321517=84, srv1927080041=43, srv27603860=59, srv1836953328=38, srv460246314=70, srv1001492600=0, srv1774339935=34, srv928364369=97, srv1838135305=39, srv368030586=64, srv304118065=61, srv232264678=56, srv2078613391=52, srv1965043102=44, srv491700988=72, srv1455809945=20, srv2081621070=53, srv827258793=92, srv1407347639=18, srv1092870270=5, srv1342760164=13, srv1785246743=35, srv418710095=65, srv348459046=63, srv2070985487=50, srv427759035=66, srv542778990=79, srv265003565=58, srv280389588=60, srv502747216=74, srv2006685054=46, srv884540661=95, srv1010735243=2, srv260383061=57, srv1200563167=10, srv2102186448=55, srv695750920=85, srv1647583933=30, srv2026262529=47, srv78555481=89, srv143242153=19, srv1049333031=4, srv546713662=80, srv100344502=1, srv181501488=36, srv435391891=68, srv771578911=87, srv991092790=99, srv1398917727=17, srv534271845=78, srv1917468271=42, srv2059944144=49, srv501675314=73, srv2034199265=48, srv2099120681=54, srv135065636=14, srv1488921574=22, srv802860922=90, srv1528669297=25, srv1674185934=31, srv986949618=98, srv1299514362=12} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-17 13:53:25,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-17 13:53:25,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-17 13:53:25,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-17 13:53:25,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-17 13:53:25,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-17 13:53:25,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-17 13:53:25,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-17 13:53:25,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-17 13:53:25,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-17 13:53:25,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-17 13:53:25,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-17 13:53:25,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-17 13:53:25,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-17 13:53:25,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-17 13:53:25,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-17 13:53:25,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-17 13:53:25,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-17 13:53:25,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-17 13:53:25,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-17 13:53:25,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-17 13:53:25,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-17 13:53:25,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-17 13:53:25,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-17 13:53:25,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-17 13:53:25,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-17 13:53:25,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-17 13:53:25,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-17 13:53:25,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-17 13:53:25,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-17 13:53:25,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-17 13:53:25,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-17 13:53:25,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-17 13:53:25,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-17 13:53:25,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-17 13:53:25,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-17 13:53:25,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-17 13:53:25,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-17 13:53:25,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-17 13:53:25,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-17 13:53:25,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-17 13:53:25,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-17 13:53:25,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-17 13:53:25,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-17 13:53:25,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-17 13:53:25,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-17 13:53:25,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-17 13:53:25,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-17 13:53:25,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-17 13:53:25,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-17 13:53:25,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-17 13:53:25,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-17 13:53:25,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-17 13:53:25,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-17 13:53:25,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-17 13:53:25,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-17 13:53:25,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-17 13:53:25,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-17 13:53:25,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-17 13:53:25,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-17 13:53:25,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-17 13:53:25,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-17 13:53:25,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-17 13:53:25,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-17 13:53:25,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-17 13:53:25,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-17 13:53:25,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-17 13:53:25,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-17 13:53:25,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-17 13:53:25,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-17 13:53:25,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-17 13:53:25,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-17 13:53:25,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-17 13:53:25,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-17 13:53:25,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-17 13:53:25,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-17 13:53:25,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-17 13:53:25,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-17 13:53:25,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-17 13:53:25,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-17 13:53:25,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-17 13:53:25,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-17 13:53:25,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-17 13:53:25,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-17 13:53:25,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-17 13:53:25,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-17 13:53:25,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-17 13:53:25,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-17 13:53:25,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-17 13:53:25,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-17 13:53:25,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-17 13:53:25,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-17 13:53:25,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-17 13:53:25,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-17 13:53:25,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-17 13:53:25,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-17 13:53:25,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-17 13:53:25,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-17 13:53:25,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-17 13:53:25,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-17 13:53:25,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-17 13:53:25,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-17 13:53:26,116 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.041044171546729395, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05333543766892307, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.041044171546729395, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17816666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24591666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-17 13:53:26,117 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-17 13:53:26,117 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18297631088294875, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.041044171546729395, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05333543766892307, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.041044171546729395, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17816666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24591666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-17 13:53:26,371 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 355 ms to try 5000 different iterations.  Found a solution that moves 2614 regions; Going from a computed imbalance of 0.18297631088294875 to a new imbalance of 0.01606115548604583. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02271507341874564); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03301191840007822, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29044444444444445, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02271507341874564); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013083333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0445, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-17 13:53:26,381 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv2059944144:79 , srv845801361:79 , srv1010735243:80 , srv1342760164:81 , srv1494559835:81 , srv986949618:81 , srv1522876137:82 , srv2070985487:82 , srv345058517:82 , srv447135110:82 , srv491700988:82 , srv348459046:83 , srv418710095:83 , srv427759035:83 , srv73738423:83 , srv1001492600:84 , srv100344502:84 , srv1165373178:84 , srv1287536769:84 , srv1528669297:84 , srv1770440135:84 , srv1874064991:84 , srv2099120681:84 , srv304118065:84 , srv460815577:84 , srv502747216:84 , srv528342269:84 , srv1043443795:85 , srv1092870270:85 , srv1200563167:85 , srv1370707343:85 , srv1382806506:85 , srv1407347639:85 , srv143242153:85 , srv1455809945:85 , srv1647583933:85 , srv181501488:85 , srv1838135305:85 , srv2034199265:85 , srv207102330:85 , srv434850845:85 , srv435391891:85 , srv460246314:85 , srv501675314:85 , srv534271845:85 , srv546713662:85 , srv631278452:85 , srv686321517:85 , srv778908639:85 , srv827258793:85 , srv1181199171:86 , srv1398917727:86 , srv1767070965:86 , srv1774339935:86 , srv1785246743:86 , srv2078613391:86 , srv260383061:86 , srv368030586:86 , srv522004608:86 , srv771578911:86 , srv78555481:86 , srv991092790:86 , srv118276342:87 , srv1614094334:87 , srv1639131352:87 , srv1836953328:87 , srv1965043102:87 , srv1975685238:87 , srv2081621070:87 , srv265003565:87 , srv672211332:87 , srv832948629:87 , srv884540661:87 , srv928364369:87 , srv1049333031:88 , srv135065636:88 , srv1488921574:88 , srv1542458070:88 , srv1595555996:88 , srv1674185934:88 , srv1917468271:88 , srv1927080041:88 , srv2006685054:88 , srv2026262529:88 , srv2102186448:88 , srv27603860:88 , srv280389588:88 , srv520172517:88 , srv695750920:88 , srv802860922:88 , srv890286880:88 , srv148464192:89 , srv232264678:89 , srv542778990:89 , srv1833889580:90 , srv1845849854:90 , srv1099533769:91 , srv1299514362:91 , srv825111372:92 , srv610030727:521 }
2023-10-17 13:53:26,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-17 13:53:26,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1043443795=3, srv1595555996=27, srv148464192=21, srv1614094334=28, srv610030727=81, srv1370707343=15, srv73738423=86, srv1181199171=8, srv631278452=82, srv672211332=83, srv845801361=94, srv1165373178=7, srv520172517=75, srv1639131352=29, srv1494559835=23, srv447135110=69, srv1287536769=11, srv832948629=93, srv522004608=76, srv1833889580=37, srv345058517=62, srv528342269=77, srv1099533769=6, srv1542458070=26, srv207102330=51, srv460815577=71, srv890286880=96, srv118276342=9, srv825111372=91, srv1522876137=24, srv1770440135=33, srv1382806506=16, srv1975685238=45, srv1767070965=32, srv1845849854=40, srv778908639=88, srv434850845=67, srv1874064991=41, srv686321517=84, srv1927080041=43, srv27603860=59, srv1836953328=38, srv460246314=70, srv1001492600=0, srv1774339935=34, srv928364369=97, srv1838135305=39, srv368030586=64, srv304118065=61, srv232264678=56, srv2078613391=52, srv1965043102=44, srv491700988=72, srv1455809945=20, srv2081621070=53, srv827258793=92, srv1407347639=18, srv1092870270=5, srv1342760164=13, srv1785246743=35, srv418710095=65, srv348459046=63, srv2070985487=50, srv427759035=66, srv542778990=79, srv265003565=58, srv280389588=60, srv502747216=74, srv2006685054=46, srv884540661=95, srv1010735243=2, srv260383061=57, srv1200563167=10, srv2102186448=55, srv695750920=85, srv1647583933=30, srv2026262529=47, srv78555481=89, srv143242153=19, srv1049333031=4, srv546713662=80, srv100344502=1, srv181501488=36, srv435391891=68, srv771578911=87, srv991092790=99, srv1398917727=17, srv534271845=78, srv1917468271=42, srv2059944144=49, srv501675314=73, srv2034199265=48, srv2099120681=54, srv135065636=14, srv1488921574=22, srv802860922=90, srv1528669297=25, srv1674185934=31, srv986949618=98, srv1299514362=12} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-17 13:53:26,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-17 13:53:26,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-17 13:53:26,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-17 13:53:26,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-17 13:53:26,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-17 13:53:26,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-17 13:53:26,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-17 13:53:26,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-17 13:53:26,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-17 13:53:26,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-17 13:53:26,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-17 13:53:26,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-17 13:53:26,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-17 13:53:26,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-17 13:53:26,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-17 13:53:26,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-17 13:53:26,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-17 13:53:26,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-17 13:53:26,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-17 13:53:26,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-17 13:53:26,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-17 13:53:26,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-17 13:53:26,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-17 13:53:26,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-17 13:53:26,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-17 13:53:26,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-17 13:53:26,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-17 13:53:26,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-17 13:53:26,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-17 13:53:26,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-17 13:53:26,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-17 13:53:26,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-17 13:53:26,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-17 13:53:26,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-17 13:53:26,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-17 13:53:26,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-17 13:53:26,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-17 13:53:26,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-17 13:53:26,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-17 13:53:26,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-17 13:53:26,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-17 13:53:26,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-17 13:53:26,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-17 13:53:26,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-17 13:53:26,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-17 13:53:26,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-17 13:53:26,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-17 13:53:26,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-17 13:53:26,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-17 13:53:26,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-17 13:53:26,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-17 13:53:26,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-17 13:53:26,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-17 13:53:26,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-17 13:53:26,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-17 13:53:26,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-17 13:53:26,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-17 13:53:26,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-17 13:53:26,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-17 13:53:26,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-17 13:53:26,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-17 13:53:26,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-17 13:53:26,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-17 13:53:26,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-17 13:53:26,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-17 13:53:26,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-17 13:53:26,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-17 13:53:26,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-17 13:53:26,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-17 13:53:26,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-17 13:53:26,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-17 13:53:26,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-17 13:53:26,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-17 13:53:26,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-17 13:53:26,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-17 13:53:26,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-17 13:53:26,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-17 13:53:26,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-17 13:53:26,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-17 13:53:26,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-17 13:53:26,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-17 13:53:26,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-17 13:53:26,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-17 13:53:26,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-17 13:53:26,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-17 13:53:26,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-17 13:53:26,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-17 13:53:26,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-17 13:53:26,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-17 13:53:26,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-17 13:53:26,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-17 13:53:26,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-17 13:53:26,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-17 13:53:26,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-17 13:53:26,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-17 13:53:26,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-17 13:53:26,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-17 13:53:26,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-17 13:53:26,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-17 13:53:26,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-17 13:53:26,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-17 13:53:26,535 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02271507341874564); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03301191840007822, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02271507341874564); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013083333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0445, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-17 13:53:26,536 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-17 13:53:26,536 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.016042851866834733, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02271507341874564); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03301191840007822, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02271507341874564); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013083333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0445, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-17 13:53:26,784 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 344 ms to try 5000 different iterations.  Found a solution that moves 1029 regions; Going from a computed imbalance of 0.016042851866834733 to a new imbalance of 8.509904814963356E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005540945998106677); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004854793875029461); MoveCostFunction : (multiplier=7.0, imbalance=0.11433333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005540945998106677); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.0E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-17 13:53:26,786 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv148464192:83 , srv1833889580:83 , srv1836953328:83 , srv1927080041:83 , srv522004608:83 , srv1043443795:84 , srv1049333031:84 , srv1099533769:84 , srv1299514362:84 , srv1382806506:84 , srv1398917727:84 , srv1542458070:84 , srv1614094334:84 , srv1785246743:84 , srv181501488:84 , srv1965043102:84 , srv2070985487:84 , srv2081621070:84 , srv2099120681:84 , srv265003565:84 , srv27603860:84 , srv348459046:84 , srv825111372:84 , srv1287536769:85 , srv135065636:85 , srv1522876137:85 , srv1528669297:85 , srv1639131352:85 , srv1767070965:85 , srv1774339935:85 , srv1845849854:85 , srv1917468271:85 , srv2059944144:85 , srv232264678:85 , srv260383061:85 , srv280389588:85 , srv304118065:85 , srv528342269:85 , srv542778990:85 , srv778908639:85 , srv802860922:85 , srv845801361:85 , srv884540661:85 , srv890286880:85 , srv928364369:85 , srv986949618:85 , srv100344502:86 , srv1165373178:86 , srv118276342:86 , srv1200563167:86 , srv1370707343:86 , srv1407347639:86 , srv1455809945:86 , srv1488921574:86 , srv1494559835:86 , srv1595555996:86 , srv1674185934:86 , srv1838135305:86 , srv1975685238:86 , srv2026262529:86 , srv2034199265:86 , srv2102186448:86 , srv368030586:86 , srv418710095:86 , srv435391891:86 , srv447135110:86 , srv460246314:86 , srv501675314:86 , srv520172517:86 , srv672211332:86 , srv686321517:86 , srv695750920:86 , srv771578911:86 , srv78555481:86 , srv827258793:86 , srv832948629:86 , srv1001492600:87 , srv1092870270:87 , srv1342760164:87 , srv143242153:87 , srv1647583933:87 , srv1770440135:87 , srv1874064991:87 , srv2006685054:87 , srv207102330:87 , srv345058517:87 , srv434850845:87 , srv460815577:87 , srv491700988:87 , srv534271845:87 , srv546713662:87 , srv631278452:87 , srv991092790:87 , srv1010735243:88 , srv1181199171:88 , srv2078613391:88 , srv427759035:88 , srv502747216:88 , srv73738423:88 , srv610030727:531 }
2023-10-17 13:53:26,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-17 13:53:26,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1043443795=3, srv1595555996=27, srv148464192=21, srv1614094334=28, srv610030727=81, srv1370707343=15, srv73738423=86, srv1181199171=8, srv631278452=82, srv672211332=83, srv845801361=94, srv1165373178=7, srv520172517=75, srv1639131352=29, srv1494559835=23, srv447135110=69, srv1287536769=11, srv832948629=93, srv522004608=76, srv1833889580=37, srv345058517=62, srv528342269=77, srv1099533769=6, srv1542458070=26, srv207102330=51, srv460815577=71, srv890286880=96, srv118276342=9, srv825111372=91, srv1522876137=24, srv1770440135=33, srv1382806506=16, srv1975685238=45, srv1767070965=32, srv1845849854=40, srv778908639=88, srv434850845=67, srv1874064991=41, srv686321517=84, srv1927080041=43, srv27603860=59, srv1836953328=38, srv460246314=70, srv1001492600=0, srv1774339935=34, srv928364369=97, srv1838135305=39, srv368030586=64, srv304118065=61, srv232264678=56, srv2078613391=52, srv1965043102=44, srv491700988=72, srv1455809945=20, srv2081621070=53, srv827258793=92, srv1407347639=18, srv1092870270=5, srv1342760164=13, srv1785246743=35, srv418710095=65, srv348459046=63, srv2070985487=50, srv427759035=66, srv542778990=79, srv265003565=58, srv280389588=60, srv502747216=74, srv2006685054=46, srv884540661=95, srv1010735243=2, srv260383061=57, srv1200563167=10, srv2102186448=55, srv695750920=85, srv1647583933=30, srv2026262529=47, srv78555481=89, srv143242153=19, srv1049333031=4, srv546713662=80, srv100344502=1, srv181501488=36, srv435391891=68, srv771578911=87, srv991092790=99, srv1398917727=17, srv534271845=78, srv1917468271=42, srv2059944144=49, srv501675314=73, srv2034199265=48, srv2099120681=54, srv135065636=14, srv1488921574=22, srv802860922=90, srv1528669297=25, srv1674185934=31, srv986949618=98, srv1299514362=12} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-17 13:53:26,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-17 13:53:26,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-17 13:53:26,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-17 13:53:26,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-17 13:53:26,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-17 13:53:26,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-17 13:53:26,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-17 13:53:26,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-17 13:53:26,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-17 13:53:26,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-17 13:53:26,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-17 13:53:26,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-17 13:53:26,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-17 13:53:26,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-17 13:53:26,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-17 13:53:26,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-17 13:53:26,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-17 13:53:26,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-17 13:53:26,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-17 13:53:26,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-17 13:53:26,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-17 13:53:26,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-17 13:53:26,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-17 13:53:26,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-17 13:53:26,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-17 13:53:26,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-17 13:53:26,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-17 13:53:26,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-17 13:53:26,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-17 13:53:26,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-17 13:53:26,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-17 13:53:26,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-17 13:53:26,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-17 13:53:26,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-17 13:53:26,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-17 13:53:26,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-17 13:53:26,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-17 13:53:26,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-17 13:53:26,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-17 13:53:26,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-17 13:53:26,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-17 13:53:26,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-17 13:53:26,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-17 13:53:26,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-17 13:53:26,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-17 13:53:26,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-17 13:53:26,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-17 13:53:26,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-17 13:53:26,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-17 13:53:26,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-17 13:53:26,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-17 13:53:26,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-17 13:53:26,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-17 13:53:26,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-17 13:53:26,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-17 13:53:26,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-17 13:53:26,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-17 13:53:26,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-17 13:53:26,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-17 13:53:26,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-17 13:53:26,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-17 13:53:26,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-17 13:53:26,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-17 13:53:26,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-17 13:53:26,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-17 13:53:26,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-17 13:53:26,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-17 13:53:26,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-17 13:53:26,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-17 13:53:26,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-17 13:53:26,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-17 13:53:26,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-17 13:53:26,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-17 13:53:26,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-17 13:53:26,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-17 13:53:26,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-17 13:53:26,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-17 13:53:26,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-17 13:53:26,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-17 13:53:26,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-17 13:53:26,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-17 13:53:26,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-17 13:53:26,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-17 13:53:26,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-17 13:53:26,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-17 13:53:26,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-17 13:53:26,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-17 13:53:26,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-17 13:53:26,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-17 13:53:26,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-17 13:53:26,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-17 13:53:26,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-17 13:53:26,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-17 13:53:26,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-17 13:53:26,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-17 13:53:26,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-17 13:53:26,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-17 13:53:26,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-17 13:53:26,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-17 13:53:26,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-17 13:53:26,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-17 13:53:26,923 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005540945998106677); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004854793875029461); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005540945998106677); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.0E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-17 13:53:26,923 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-17 13:53:26,923 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=8.437852694962516E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005540945998106677); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004854793875029461); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005540945998106677); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.0E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-17 13:53:27,307 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 481 ms to try 5000 different iterations.  Found a solution that moves 237 regions; Going from a computed imbalance of 8.437852694962516E-4 to a new imbalance of 1.8701011593974024E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0016412198797244363); MoveCostFunction : (multiplier=7.0, imbalance=0.026333333333333334, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-17 13:53:27,308 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv491700988:82 , srv1370707343:83 , srv368030586:83 , srv447135110:83 , srv502747216:83 , srv802860922:83 , srv1001492600:84 , srv1010735243:84 , srv1092870270:84 , srv1181199171:84 , srv118276342:84 , srv1342760164:84 , srv143242153:84 , srv148464192:84 , srv1488921574:84 , srv1494559835:84 , srv1542458070:84 , srv1595555996:84 , srv1614094334:84 , srv1647583933:84 , srv1770440135:84 , srv1774339935:84 , srv1785246743:84 , srv1836953328:84 , srv1845849854:84 , srv1874064991:84 , srv1917468271:84 , srv1965043102:84 , srv2006685054:84 , srv2034199265:84 , srv2078613391:84 , srv2081621070:84 , srv2102186448:84 , srv280389588:84 , srv304118065:84 , srv345058517:84 , srv348459046:84 , srv418710095:84 , srv434850845:84 , srv520172517:84 , srv534271845:84 , srv542778990:84 , srv546713662:84 , srv695750920:84 , srv73738423:84 , srv825111372:84 , srv827258793:84 , srv890286880:84 , srv986949618:84 , srv100344502:85 , srv1043443795:85 , srv1049333031:85 , srv1099533769:85 , srv1165373178:85 , srv1200563167:85 , srv1287536769:85 , srv1299514362:85 , srv1382806506:85 , srv1398917727:85 , srv1407347639:85 , srv1455809945:85 , srv1528669297:85 , srv1639131352:85 , srv1674185934:85 , srv1767070965:85 , srv181501488:85 , srv1833889580:85 , srv1927080041:85 , srv1975685238:85 , srv2026262529:85 , srv2059944144:85 , srv207102330:85 , srv2099120681:85 , srv232264678:85 , srv260383061:85 , srv265003565:85 , srv27603860:85 , srv435391891:85 , srv460246314:85 , srv460815577:85 , srv501675314:85 , srv522004608:85 , srv528342269:85 , srv631278452:85 , srv686321517:85 , srv771578911:85 , srv778908639:85 , srv78555481:85 , srv832948629:85 , srv845801361:85 , srv884540661:85 , srv928364369:85 , srv991092790:85 , srv135065636:86 , srv1522876137:86 , srv1838135305:86 , srv2070985487:86 , srv427759035:86 , srv672211332:86 , srv610030727:635 }
2023-10-17 13:53:27,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-17 13:53:27,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1043443795=3, srv1595555996=27, srv148464192=21, srv1614094334=28, srv610030727=81, srv1370707343=15, srv73738423=86, srv1181199171=8, srv631278452=82, srv672211332=83, srv845801361=94, srv1165373178=7, srv520172517=75, srv1639131352=29, srv1494559835=23, srv447135110=69, srv1287536769=11, srv832948629=93, srv522004608=76, srv1833889580=37, srv345058517=62, srv528342269=77, srv1099533769=6, srv1542458070=26, srv207102330=51, srv460815577=71, srv890286880=96, srv118276342=9, srv825111372=91, srv1522876137=24, srv1770440135=33, srv1382806506=16, srv1975685238=45, srv1767070965=32, srv1845849854=40, srv778908639=88, srv434850845=67, srv1874064991=41, srv686321517=84, srv1927080041=43, srv27603860=59, srv1836953328=38, srv460246314=70, srv1001492600=0, srv1774339935=34, srv928364369=97, srv1838135305=39, srv368030586=64, srv304118065=61, srv232264678=56, srv2078613391=52, srv1965043102=44, srv491700988=72, srv1455809945=20, srv2081621070=53, srv827258793=92, srv1407347639=18, srv1092870270=5, srv1342760164=13, srv1785246743=35, srv418710095=65, srv348459046=63, srv2070985487=50, srv427759035=66, srv542778990=79, srv265003565=58, srv280389588=60, srv502747216=74, srv2006685054=46, srv884540661=95, srv1010735243=2, srv260383061=57, srv1200563167=10, srv2102186448=55, srv695750920=85, srv1647583933=30, srv2026262529=47, srv78555481=89, srv143242153=19, srv1049333031=4, srv546713662=80, srv100344502=1, srv181501488=36, srv435391891=68, srv771578911=87, srv991092790=99, srv1398917727=17, srv534271845=78, srv1917468271=42, srv2059944144=49, srv501675314=73, srv2034199265=48, srv2099120681=54, srv135065636=14, srv1488921574=22, srv802860922=90, srv1528669297=25, srv1674185934=31, srv986949618=98, srv1299514362=12} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-17 13:53:27,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-17 13:53:27,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-17 13:53:27,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-17 13:53:27,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-17 13:53:27,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-17 13:53:27,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-17 13:53:27,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-17 13:53:27,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-17 13:53:27,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-17 13:53:27,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-17 13:53:27,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-17 13:53:27,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-17 13:53:27,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-17 13:53:27,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-17 13:53:27,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-17 13:53:27,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-17 13:53:27,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-17 13:53:27,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-17 13:53:27,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-17 13:53:27,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-17 13:53:27,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-17 13:53:27,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-17 13:53:27,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-17 13:53:27,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-17 13:53:27,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-17 13:53:27,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-17 13:53:27,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-17 13:53:27,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-17 13:53:27,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-17 13:53:27,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-17 13:53:27,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-17 13:53:27,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-17 13:53:27,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-17 13:53:27,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-17 13:53:27,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-17 13:53:27,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-17 13:53:27,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-17 13:53:27,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-17 13:53:27,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-17 13:53:27,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-17 13:53:27,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-17 13:53:27,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-17 13:53:27,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-17 13:53:27,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-17 13:53:27,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-17 13:53:27,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-17 13:53:27,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-17 13:53:27,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-17 13:53:27,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-17 13:53:27,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-17 13:53:27,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-17 13:53:27,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-17 13:53:27,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-17 13:53:27,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-17 13:53:27,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-17 13:53:27,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-17 13:53:27,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-17 13:53:27,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-17 13:53:27,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-17 13:53:27,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-17 13:53:27,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-17 13:53:27,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-17 13:53:27,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-17 13:53:27,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-17 13:53:27,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-17 13:53:27,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-17 13:53:27,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-17 13:53:27,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-17 13:53:27,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-17 13:53:27,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-17 13:53:27,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-17 13:53:27,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-17 13:53:27,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-17 13:53:27,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-17 13:53:27,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-17 13:53:27,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-17 13:53:27,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-17 13:53:27,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-17 13:53:27,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-17 13:53:27,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-17 13:53:27,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-17 13:53:27,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-17 13:53:27,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-17 13:53:27,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-17 13:53:27,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-17 13:53:27,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-17 13:53:27,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-17 13:53:27,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-17 13:53:27,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-17 13:53:27,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-17 13:53:27,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-17 13:53:27,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-17 13:53:27,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-17 13:53:27,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-17 13:53:27,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-17 13:53:27,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-17 13:53:27,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-17 13:53:27,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-17 13:53:27,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-17 13:53:27,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-17 13:53:27,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-17 13:53:27,467 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.7041502124566916E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0016412198797244363); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-17 13:53:27,467 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv491700988:82 , srv1370707343:83 , srv368030586:83 , srv447135110:83 , srv502747216:83 , srv802860922:83 , srv1001492600:84 , srv1010735243:84 , srv1092870270:84 , srv1181199171:84 , srv118276342:84 , srv1342760164:84 , srv143242153:84 , srv148464192:84 , srv1488921574:84 , srv1494559835:84 , srv1542458070:84 , srv1595555996:84 , srv1614094334:84 , srv1647583933:84 , srv1770440135:84 , srv1774339935:84 , srv1785246743:84 , srv1836953328:84 , srv1845849854:84 , srv1874064991:84 , srv1917468271:84 , srv1965043102:84 , srv2006685054:84 , srv2034199265:84 , srv2078613391:84 , srv2081621070:84 , srv2102186448:84 , srv280389588:84 , srv304118065:84 , srv345058517:84 , srv348459046:84 , srv418710095:84 , srv434850845:84 , srv520172517:84 , srv534271845:84 , srv542778990:84 , srv546713662:84 , srv695750920:84 , srv73738423:84 , srv825111372:84 , srv827258793:84 , srv890286880:84 , srv986949618:84 , srv100344502:85 , srv1043443795:85 , srv1049333031:85 , srv1099533769:85 , srv1165373178:85 , srv1200563167:85 , srv1287536769:85 , srv1299514362:85 , srv1382806506:85 , srv1398917727:85 , srv1407347639:85 , srv1455809945:85 , srv1528669297:85 , srv1639131352:85 , srv1674185934:85 , srv1767070965:85 , srv181501488:85 , srv1833889580:85 , srv1927080041:85 , srv1975685238:85 , srv2026262529:85 , srv2059944144:85 , srv207102330:85 , srv2099120681:85 , srv232264678:85 , srv260383061:85 , srv265003565:85 , srv27603860:85 , srv435391891:85 , srv460246314:85 , srv460815577:85 , srv501675314:85 , srv522004608:85 , srv528342269:85 , srv631278452:85 , srv686321517:85 , srv771578911:85 , srv778908639:85 , srv78555481:85 , srv832948629:85 , srv845801361:85 , srv884540661:85 , srv928364369:85 , srv991092790:85 , srv135065636:86 , srv1522876137:86 , srv1838135305:86 , srv2070985487:86 , srv427759035:86 , srv672211332:86 , srv610030727:635 }
2023-10-17 13:53:27,505 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=304 (was 304), ProcessCount=169 (was 169), AvailableMemoryMB=7614 (was 8168)
2023-10-17 13:53:27,515 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=304, ProcessCount=169, AvailableMemoryMB=7612
2023-10-17 13:53:27,515 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-10-17 13:53:27,516 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-10-17 13:53:27,523 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1181521980:84 , srv1200626333:84 , srv1320173504:84 , srv1391493031:84 , srv1392167491:84 , srv1469436958:84 , srv1565545948:84 , srv158982943:84 , srv1718840843:84 , srv1738810720:84 , srv1805940395:84 , srv1932204621:84 , srv196882662:84 , srv1971932823:84 , srv1991171917:84 , srv2061562917:84 , srv2115862942:84 , srv263048695:84 , srv30759252:84 , srv393942320:84 , srv39576957:84 , srv447508956:84 , srv664389061:84 , srv706826052:84 , srv707823613:84 , srv718800308:84 , srv724577254:84 , srv798732237:84 , srv973809671:84 , srv632929774:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-10-17 13:53:27,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-17 13:53:27,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1200626333=1, srv39576957=20, srv1565545948=6, srv30759252=18, srv393942320=19, srv798732237=28, srv1391493031=3, srv1392167491=4, srv706826052=24, srv1971932823=13, srv447508956=21, srv1320173504=2, srv1718840843=8, srv1805940395=10, srv2061562917=15, srv973809671=29, srv158982943=7, srv1181521980=0, srv2115862942=16, srv1932204621=11, srv263048695=17, srv707823613=25, srv196882662=12, srv632929774=22, srv1738810720=9, srv1991171917=14, srv718800308=26, srv664389061=23, srv724577254=27, srv1469436958=5} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-17 13:53:27,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-17 13:53:27,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-17 13:53:27,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-17 13:53:27,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-17 13:53:27,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-17 13:53:27,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-17 13:53:27,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-17 13:53:27,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-17 13:53:27,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-17 13:53:27,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-17 13:53:27,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-17 13:53:27,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-17 13:53:27,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-17 13:53:27,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-17 13:53:27,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-17 13:53:27,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-17 13:53:27,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-17 13:53:27,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-17 13:53:27,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-17 13:53:27,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-17 13:53:27,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-17 13:53:27,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-17 13:53:27,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-17 13:53:27,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-17 13:53:27,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-17 13:53:27,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-17 13:53:27,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-17 13:53:27,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-17 13:53:27,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-17 13:53:27,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-17 13:53:27,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-10-17 13:53:27,550 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.080325140695319, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-17 13:53:27,550 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912446145159755, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.080325140695319, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-10-17 13:55:27,538 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6305273 different iterations.  Found a solution that moves 1855 regions; Going from a computed imbalance of 0.9912446145159755 to a new imbalance of 4.329662539733032E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.687037037037037, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-17 13:55:27,546 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1181521980:90 , srv1200626333:90 , srv1320173504:90 , srv1391493031:90 , srv1392167491:90 , srv1469436958:90 , srv1565545948:90 , srv158982943:90 , srv1718840843:90 , srv1738810720:90 , srv1805940395:90 , srv1932204621:90 , srv196882662:90 , srv1971932823:90 , srv1991171917:90 , srv2061562917:90 , srv2115862942:90 , srv263048695:90 , srv30759252:90 , srv393942320:90 , srv39576957:90 , srv447508956:90 , srv632929774:90 , srv664389061:90 , srv706826052:90 , srv707823613:90 , srv718800308:90 , srv724577254:90 , srv798732237:90 , srv973809671:90 }
2023-10-17 13:55:27,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-17 13:55:27,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1200626333=1, srv39576957=20, srv1565545948=6, srv30759252=18, srv393942320=19, srv798732237=28, srv1391493031=3, srv1392167491=4, srv706826052=24, srv1971932823=13, srv447508956=21, srv1320173504=2, srv1718840843=8, srv1805940395=10, srv2061562917=15, srv973809671=29, srv158982943=7, srv1181521980=0, srv2115862942=16, srv1932204621=11, srv263048695=17, srv707823613=25, srv196882662=12, srv632929774=22, srv1738810720=9, srv1991171917=14, srv718800308=26, srv664389061=23, srv724577254=27, srv1469436958=5} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-17 13:55:27,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-17 13:55:27,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-17 13:55:27,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-17 13:55:27,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-17 13:55:27,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-17 13:55:27,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-17 13:55:27,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-17 13:55:27,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-17 13:55:27,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-17 13:55:27,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-17 13:55:27,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-17 13:55:27,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-17 13:55:27,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-17 13:55:27,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-17 13:55:27,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-17 13:55:27,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-17 13:55:27,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-17 13:55:27,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-17 13:55:27,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-17 13:55:27,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-17 13:55:27,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-17 13:55:27,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-17 13:55:27,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-17 13:55:27,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-17 13:55:27,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-17 13:55:27,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-17 13:55:27,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-17 13:55:27,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-17 13:55:27,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-17 13:55:27,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-17 13:55:27,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-10-17 13:55:27,565 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-17 13:55:27,566 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1181521980:90 , srv1200626333:90 , srv1320173504:90 , srv1391493031:90 , srv1392167491:90 , srv1469436958:90 , srv1565545948:90 , srv158982943:90 , srv1718840843:90 , srv1738810720:90 , srv1805940395:90 , srv1932204621:90 , srv196882662:90 , srv1971932823:90 , srv1991171917:90 , srv2061562917:90 , srv2115862942:90 , srv263048695:90 , srv30759252:90 , srv393942320:90 , srv39576957:90 , srv447508956:90 , srv632929774:90 , srv664389061:90 , srv706826052:90 , srv707823613:90 , srv718800308:90 , srv724577254:90 , srv798732237:90 , srv973809671:90 }
2023-10-17 13:55:27,578 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=283 (was 304), ProcessCount=167 (was 169), AvailableMemoryMB=7756 (was 7612) - AvailableMemoryMB LEAK? -