2023-03-20 10:59:28,399 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-03-20 10:59:28,798 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-03-20 10:59:28,922 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-03-20 10:59:29,006 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-20 10:59:29,016 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-03-20 10:59:29,035 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=744, ProcessCount=183, AvailableMemoryMB=11168
2023-03-20 10:59:29,038 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-20 10:59:29,039 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-03-20 10:59:29,227 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1055799241:84 , srv107604990:84 , srv1089714351:84 , srv1113817638:84 , srv1158012633:84 , srv1172044630:84 , srv118799377:84 , srv1191705799:84 , srv1205914381:84 , srv1237071483:84 , srv123865504:84 , srv1248938040:84 , srv1274351127:84 , srv1345565549:84 , srv1365536363:84 , srv1366038401:84 , srv1382311883:84 , srv1387273818:84 , srv1405171448:84 , srv1437552599:84 , srv1480391018:84 , srv1501927064:84 , srv1518986398:84 , srv1535035759:84 , srv1538697825:84 , srv158119382:84 , srv1590947448:84 , srv1611603252:84 , srv1615442631:84 , srv1635504608:84 , srv1659369709:84 , srv1693002756:84 , srv1704743515:84 , srv172990379:84 , srv1761706814:84 , srv1780529305:84 , srv1829901522:84 , srv1859795605:84 , srv1893634580:84 , srv1919808824:84 , srv192565657:84 , srv1926750248:84 , srv1938328033:84 , srv1939039804:84 , srv1968069250:84 , srv1986312053:84 , srv1988340987:84 , srv2001065939:84 , srv200222919:84 , srv2039670153:84 , srv2053248024:84 , srv209128286:84 , srv2125154937:84 , srv218840427:84 , srv223323027:84 , srv228018928:84 , srv237575550:84 , srv245727256:84 , srv30063425:84 , srv337620324:84 , srv339037555:84 , srv35233419:84 , srv370149171:84 , srv39372642:84 , srv395322939:84 , srv407022649:84 , srv417160130:84 , srv438375813:84 , srv461527941:84 , srv468106232:84 , srv473744443:84 , srv486434451:84 , srv524189971:84 , srv529906123:84 , srv547576155:84 , srv5481193:84 , srv556050740:84 , srv566788738:84 , srv574203871:84 , srv62115494:84 , srv623392411:84 , srv642351216:84 , srv650957900:84 , srv710906040:84 , srv711687758:84 , srv739948855:84 , srv747391714:84 , srv753318800:84 , srv764446805:84 , srv784163228:84 , srv787725898:84 , srv83162835:84 , srv841768964:84 , srv845860632:84 , srv887776494:84 , srv89458795:84 , srv924581990:84 , srv937653839:84 , srv968113524:84 , srv13822424:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-03-20 10:59:29,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-20 10:59:29,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv30063425=59, srv2039670153=50, srv245727256=58, srv764446805=89, srv1538697825=25, srv529906123=74, srv1387273818=18, srv2001065939=48, srv62115494=80, srv1893634580=39, srv1693002756=32, srv1437552599=20, srv753318800=88, srv566788738=78, srv417160130=67, srv1365536363=14, srv473744443=71, srv642351216=82, srv1113817638=3, srv1859795605=38, srv123865504=10, srv887776494=95, srv1704743515=33, srv710906040=84, srv711687758=85, srv747391714=87, srv1480391018=21, srv784163228=90, srv39372642=64, srv223323027=55, srv1988340987=47, srv924581990=97, srv1659369709=31, srv35233419=62, srv1405171448=19, srv2053248024=51, srv1191705799=7, srv2125154937=53, srv200222919=49, srv937653839=98, srv1366038401=15, srv486434451=72, srv1535035759=24, srv118799377=6, srv407022649=66, srv1248938040=11, srv228018928=56, srv468106232=70, srv556050740=77, srv1237071483=9, srv1055799241=0, srv5481193=76, srv370149171=63, srv1382311883=17, srv1501927064=22, srv841768964=93, srv218840427=54, srv339037555=61, srv1926750248=42, srv172990379=34, srv1919808824=40, srv1345565549=13, srv237575550=57, srv1611603252=28, srv1968069250=45, srv192565657=41, srv1089714351=2, srv787725898=91, srv574203871=79, srv968113524=99, srv1205914381=8, srv209128286=52, srv337620324=60, srv1939039804=44, srv83162835=92, srv524189971=73, srv845860632=94, srv1274351127=12, srv1829901522=37, srv1986312053=46, srv395322939=65, srv1590947448=27, srv1780529305=36, srv739948855=86, srv13822424=16, srv158119382=26, srv461527941=69, srv89458795=96, srv1761706814=35, srv107604990=1, srv623392411=81, srv1158012633=4, srv1172044630=5, srv438375813=68, srv1635504608=30, srv547576155=75, srv1938328033=43, srv1615442631=29, srv650957900=83, srv1518986398=23} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-20 10:59:29,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-20 10:59:29,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-20 10:59:29,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-20 10:59:29,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-20 10:59:29,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-20 10:59:29,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-20 10:59:29,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-20 10:59:29,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-20 10:59:29,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-20 10:59:29,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-20 10:59:29,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-20 10:59:29,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-20 10:59:29,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-20 10:59:29,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-20 10:59:29,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-20 10:59:29,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-20 10:59:29,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-20 10:59:29,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-20 10:59:29,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-20 10:59:29,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-20 10:59:29,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-20 10:59:29,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-20 10:59:29,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-20 10:59:29,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-20 10:59:29,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-20 10:59:29,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-20 10:59:29,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-20 10:59:29,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-20 10:59:29,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-20 10:59:29,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-20 10:59:29,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-20 10:59:29,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-20 10:59:29,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-20 10:59:29,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-20 10:59:29,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-20 10:59:29,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-20 10:59:29,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-20 10:59:29,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-20 10:59:29,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-20 10:59:29,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-20 10:59:29,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-20 10:59:29,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-20 10:59:29,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-20 10:59:29,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-20 10:59:29,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-20 10:59:29,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-20 10:59:29,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-20 10:59:29,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-20 10:59:29,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-20 10:59:29,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-20 10:59:29,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-20 10:59:29,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-20 10:59:29,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-20 10:59:29,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-20 10:59:29,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-20 10:59:29,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-20 10:59:29,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-20 10:59:29,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-20 10:59:29,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-20 10:59:29,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-20 10:59:29,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-20 10:59:29,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-20 10:59:29,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-20 10:59:29,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-20 10:59:29,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-20 10:59:29,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-20 10:59:29,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-20 10:59:29,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-20 10:59:29,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-20 10:59:29,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-20 10:59:29,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-20 10:59:29,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-20 10:59:29,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-20 10:59:29,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-20 10:59:29,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-20 10:59:29,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-20 10:59:29,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-20 10:59:29,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-20 10:59:29,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-20 10:59:29,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-20 10:59:29,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-20 10:59:29,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-20 10:59:29,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-20 10:59:29,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-20 10:59:29,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-20 10:59:29,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-20 10:59:29,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-20 10:59:29,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-20 10:59:29,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-20 10:59:29,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-20 10:59:29,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-20 10:59:29,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-20 10:59:29,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-20 10:59:29,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-20 10:59:29,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-20 10:59:29,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-20 10:59:29,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-20 10:59:29,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-20 10:59:29,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-20 10:59:29,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-20 10:59:29,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-20 10:59:29,828 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-20 10:59:29,830 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-20 10:59:29,830 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-20 10:59:30,425 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 1000 ms to try 5000 different iterations.  Found a solution that moves 4637 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18462601858703914. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.042112800501516855, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.059801392204026846, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5152222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.042112800501516855, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17975, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24766666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-20 10:59:30,446 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv223323027:81 , srv650957900:81 , srv1237071483:82 , srv1405171448:82 , srv1968069250:82 , srv473744443:82 , srv547576155:82 , srv887776494:82 , srv1055799241:83 , srv118799377:83 , srv1365536363:83 , srv1382311883:83 , srv1704743515:83 , srv1893634580:83 , srv339037555:83 , srv468106232:83 , srv524189971:83 , srv556050740:83 , srv1191705799:84 , srv1248938040:84 , srv1387273818:84 , srv1538697825:84 , srv1635504608:84 , srv172990379:84 , srv200222919:84 , srv2053248024:84 , srv218840427:84 , srv237575550:84 , srv574203871:84 , srv924581990:84 , srv1113817638:85 , srv1172044630:85 , srv1829901522:85 , srv1938328033:85 , srv209128286:85 , srv228018928:85 , srv30063425:85 , srv370149171:85 , srv395322939:85 , srv417160130:85 , srv642351216:85 , srv747391714:85 , srv107604990:86 , srv1089714351:86 , srv123865504:86 , srv1274351127:86 , srv1345565549:86 , srv1366038401:86 , srv1437552599:86 , srv1518986398:86 , srv1535035759:86 , srv1926750248:86 , srv2039670153:86 , srv461527941:86 , srv5481193:86 , srv753318800:86 , srv89458795:86 , srv937653839:86 , srv1611603252:87 , srv1659369709:87 , srv1761706814:87 , srv192565657:87 , srv1988340987:87 , srv566788738:87 , srv623392411:87 , srv711687758:87 , srv739948855:87 , srv83162835:87 , srv968113524:87 , srv1158012633:88 , srv1480391018:88 , srv1590947448:88 , srv1693002756:88 , srv1859795605:88 , srv1986312053:88 , srv2001065939:88 , srv245727256:88 , srv337620324:88 , srv39372642:88 , srv407022649:88 , srv438375813:88 , srv529906123:88 , srv62115494:88 , srv710906040:88 , srv764446805:88 , srv784163228:88 , srv787725898:88 , srv1780529305:89 , srv1919808824:89 , srv1939039804:90 , srv2125154937:90 , srv1615442631:91 , srv158119382:92 , srv841768964:92 , srv1501927064:93 , srv486434451:93 , srv35233419:94 , srv845860632:96 , srv1205914381:98 , srv13822424:464 }
2023-03-20 10:59:30,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-20 10:59:30,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv30063425=59, srv2039670153=50, srv245727256=58, srv764446805=89, srv1538697825=25, srv529906123=74, srv1387273818=18, srv2001065939=48, srv62115494=80, srv1893634580=39, srv1693002756=32, srv1437552599=20, srv753318800=88, srv566788738=78, srv417160130=67, srv1365536363=14, srv473744443=71, srv642351216=82, srv1113817638=3, srv1859795605=38, srv123865504=10, srv887776494=95, srv1704743515=33, srv710906040=84, srv711687758=85, srv747391714=87, srv1480391018=21, srv784163228=90, srv39372642=64, srv223323027=55, srv1988340987=47, srv924581990=97, srv1659369709=31, srv35233419=62, srv1405171448=19, srv2053248024=51, srv1191705799=7, srv2125154937=53, srv200222919=49, srv937653839=98, srv1366038401=15, srv486434451=72, srv1535035759=24, srv118799377=6, srv407022649=66, srv1248938040=11, srv228018928=56, srv468106232=70, srv556050740=77, srv1237071483=9, srv1055799241=0, srv5481193=76, srv370149171=63, srv1382311883=17, srv1501927064=22, srv841768964=93, srv218840427=54, srv339037555=61, srv1926750248=42, srv172990379=34, srv1919808824=40, srv1345565549=13, srv237575550=57, srv1611603252=28, srv1968069250=45, srv192565657=41, srv1089714351=2, srv787725898=91, srv574203871=79, srv968113524=99, srv1205914381=8, srv209128286=52, srv337620324=60, srv1939039804=44, srv83162835=92, srv524189971=73, srv845860632=94, srv1274351127=12, srv1829901522=37, srv1986312053=46, srv395322939=65, srv1590947448=27, srv1780529305=36, srv739948855=86, srv13822424=16, srv158119382=26, srv461527941=69, srv89458795=96, srv1761706814=35, srv107604990=1, srv623392411=81, srv1158012633=4, srv1172044630=5, srv438375813=68, srv1635504608=30, srv547576155=75, srv1938328033=43, srv1615442631=29, srv650957900=83, srv1518986398=23} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-20 10:59:30,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-20 10:59:30,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-20 10:59:30,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-20 10:59:30,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-20 10:59:30,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-20 10:59:30,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-20 10:59:30,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-20 10:59:30,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-20 10:59:30,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-20 10:59:30,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-20 10:59:30,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-20 10:59:30,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-20 10:59:30,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-20 10:59:30,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-20 10:59:30,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-20 10:59:30,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-20 10:59:30,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-20 10:59:30,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-20 10:59:30,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-20 10:59:30,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-20 10:59:30,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-20 10:59:30,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-20 10:59:30,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-20 10:59:30,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-20 10:59:30,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-20 10:59:30,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-20 10:59:30,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-20 10:59:30,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-20 10:59:30,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-20 10:59:30,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-20 10:59:30,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-20 10:59:30,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-20 10:59:30,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-20 10:59:30,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-20 10:59:30,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-20 10:59:30,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-20 10:59:30,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-20 10:59:30,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-20 10:59:30,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-20 10:59:30,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-20 10:59:30,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-20 10:59:30,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-20 10:59:30,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-20 10:59:30,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-20 10:59:30,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-20 10:59:30,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-20 10:59:30,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-20 10:59:30,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-20 10:59:30,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-20 10:59:30,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-20 10:59:30,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-20 10:59:30,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-20 10:59:30,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-20 10:59:30,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-20 10:59:30,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-20 10:59:30,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-20 10:59:30,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-20 10:59:30,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-20 10:59:30,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-20 10:59:30,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-20 10:59:30,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-20 10:59:30,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-20 10:59:30,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-20 10:59:30,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-20 10:59:30,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-20 10:59:30,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-20 10:59:30,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-20 10:59:30,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-20 10:59:30,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-20 10:59:30,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-20 10:59:30,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-20 10:59:30,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-20 10:59:30,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-20 10:59:30,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-20 10:59:30,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-20 10:59:30,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-20 10:59:30,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-20 10:59:30,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-20 10:59:30,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-20 10:59:30,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-20 10:59:30,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-20 10:59:30,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-20 10:59:30,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-20 10:59:30,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-20 10:59:30,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-20 10:59:30,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-20 10:59:30,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-20 10:59:30,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-20 10:59:30,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-20 10:59:30,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-20 10:59:30,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-20 10:59:30,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-20 10:59:30,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-20 10:59:30,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-20 10:59:30,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-20 10:59:30,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-20 10:59:30,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-20 10:59:30,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-20 10:59:30,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-20 10:59:30,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-20 10:59:30,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-20 10:59:30,744 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.042112800501516855, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.059801392204026846, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.042112800501516855, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17975, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24766666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-20 10:59:30,745 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-20 10:59:30,745 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.184593549619066, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.042112800501516855, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.059801392204026846, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.042112800501516855, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17975, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24766666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-20 10:59:31,075 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 552 ms to try 5000 different iterations.  Found a solution that moves 2671 regions; Going from a computed imbalance of 0.184593549619066 to a new imbalance of 0.01599108194385864. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021872671700152592); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03992417054791989, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29677777777777775, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021872671700152592); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012833333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04591666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-20 10:59:31,090 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1205914381:75 , srv486434451:78 , srv845860632:78 , srv158119382:80 , srv1615442631:80 , srv1501927064:81 , srv1919808824:81 , srv407022649:81 , srv1480391018:82 , srv2001065939:82 , srv711687758:82 , srv841768964:82 , srv1158012633:83 , srv2039670153:83 , srv2125154937:83 , srv337620324:83 , srv1366038401:84 , srv1590947448:84 , srv1611603252:84 , srv1939039804:84 , srv461527941:84 , srv623392411:84 , srv710906040:84 , srv739948855:84 , srv753318800:84 , srv1089714351:85 , srv1274351127:85 , srv1345565549:85 , srv1437552599:85 , srv1761706814:85 , srv1829901522:85 , srv192565657:85 , srv1926750248:85 , srv1986312053:85 , srv209128286:85 , srv35233419:85 , srv39372642:85 , srv395322939:85 , srv438375813:85 , srv5481193:85 , srv62115494:85 , srv764446805:85 , srv784163228:85 , srv787725898:85 , srv83162835:85 , srv924581990:85 , srv968113524:85 , srv123865504:86 , srv1387273818:86 , srv1859795605:86 , srv1938328033:86 , srv1988340987:86 , srv200222919:86 , srv237575550:86 , srv30063425:86 , srv339037555:86 , srv417160130:86 , srv574203871:86 , srv747391714:86 , srv937653839:86 , srv1113817638:87 , srv1518986398:87 , srv1659369709:87 , srv1693002756:87 , srv1780529305:87 , srv1893634580:87 , srv2053248024:87 , srv228018928:87 , srv245727256:87 , srv529906123:87 , srv566788738:87 , srv1055799241:88 , srv107604990:88 , srv1191705799:88 , srv1405171448:88 , srv1635504608:88 , srv218840427:88 , srv468106232:88 , srv887776494:88 , srv1172044630:89 , srv1538697825:89 , srv1704743515:89 , srv172990379:89 , srv223323027:89 , srv370149171:89 , srv473744443:89 , srv556050740:89 , srv89458795:89 , srv118799377:90 , srv1237071483:90 , srv1535035759:90 , srv1968069250:90 , srv524189971:90 , srv547576155:90 , srv1248938040:91 , srv1365536363:91 , srv1382311883:91 , srv650957900:91 , srv642351216:92 , srv13822424:504 }
2023-03-20 10:59:31,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-20 10:59:31,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv30063425=59, srv2039670153=50, srv245727256=58, srv764446805=89, srv1538697825=25, srv529906123=74, srv1387273818=18, srv2001065939=48, srv62115494=80, srv1893634580=39, srv1693002756=32, srv1437552599=20, srv753318800=88, srv566788738=78, srv417160130=67, srv1365536363=14, srv473744443=71, srv642351216=82, srv1113817638=3, srv1859795605=38, srv123865504=10, srv887776494=95, srv1704743515=33, srv710906040=84, srv711687758=85, srv747391714=87, srv1480391018=21, srv784163228=90, srv39372642=64, srv223323027=55, srv1988340987=47, srv924581990=97, srv1659369709=31, srv35233419=62, srv1405171448=19, srv2053248024=51, srv1191705799=7, srv2125154937=53, srv200222919=49, srv937653839=98, srv1366038401=15, srv486434451=72, srv1535035759=24, srv118799377=6, srv407022649=66, srv1248938040=11, srv228018928=56, srv468106232=70, srv556050740=77, srv1237071483=9, srv1055799241=0, srv5481193=76, srv370149171=63, srv1382311883=17, srv1501927064=22, srv841768964=93, srv218840427=54, srv339037555=61, srv1926750248=42, srv172990379=34, srv1919808824=40, srv1345565549=13, srv237575550=57, srv1611603252=28, srv1968069250=45, srv192565657=41, srv1089714351=2, srv787725898=91, srv574203871=79, srv968113524=99, srv1205914381=8, srv209128286=52, srv337620324=60, srv1939039804=44, srv83162835=92, srv524189971=73, srv845860632=94, srv1274351127=12, srv1829901522=37, srv1986312053=46, srv395322939=65, srv1590947448=27, srv1780529305=36, srv739948855=86, srv13822424=16, srv158119382=26, srv461527941=69, srv89458795=96, srv1761706814=35, srv107604990=1, srv623392411=81, srv1158012633=4, srv1172044630=5, srv438375813=68, srv1635504608=30, srv547576155=75, srv1938328033=43, srv1615442631=29, srv650957900=83, srv1518986398=23} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-20 10:59:31,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-20 10:59:31,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-20 10:59:31,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-20 10:59:31,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-20 10:59:31,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-20 10:59:31,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-20 10:59:31,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-20 10:59:31,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-20 10:59:31,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-20 10:59:31,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-20 10:59:31,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-20 10:59:31,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-20 10:59:31,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-20 10:59:31,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-20 10:59:31,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-20 10:59:31,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-20 10:59:31,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-20 10:59:31,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-20 10:59:31,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-20 10:59:31,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-20 10:59:31,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-20 10:59:31,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-20 10:59:31,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-20 10:59:31,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-20 10:59:31,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-20 10:59:31,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-20 10:59:31,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-20 10:59:31,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-20 10:59:31,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-20 10:59:31,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-20 10:59:31,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-20 10:59:31,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-20 10:59:31,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-20 10:59:31,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-20 10:59:31,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-20 10:59:31,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-20 10:59:31,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-20 10:59:31,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-20 10:59:31,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-20 10:59:31,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-20 10:59:31,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-20 10:59:31,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-20 10:59:31,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-20 10:59:31,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-20 10:59:31,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-20 10:59:31,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-20 10:59:31,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-20 10:59:31,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-20 10:59:31,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-20 10:59:31,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-20 10:59:31,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-20 10:59:31,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-20 10:59:31,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-20 10:59:31,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-20 10:59:31,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-20 10:59:31,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-20 10:59:31,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-20 10:59:31,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-20 10:59:31,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-20 10:59:31,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-20 10:59:31,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-20 10:59:31,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-20 10:59:31,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-20 10:59:31,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-20 10:59:31,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-20 10:59:31,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-20 10:59:31,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-20 10:59:31,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-20 10:59:31,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-20 10:59:31,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-20 10:59:31,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-20 10:59:31,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-20 10:59:31,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-20 10:59:31,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-20 10:59:31,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-20 10:59:31,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-20 10:59:31,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-20 10:59:31,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-20 10:59:31,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-20 10:59:31,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-20 10:59:31,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-20 10:59:31,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-20 10:59:31,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-20 10:59:31,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-20 10:59:31,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-20 10:59:31,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-20 10:59:31,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-20 10:59:31,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-20 10:59:31,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-20 10:59:31,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-20 10:59:31,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-20 10:59:31,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-20 10:59:31,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-20 10:59:31,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-20 10:59:31,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-20 10:59:31,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-20 10:59:31,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-20 10:59:31,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-20 10:59:31,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-20 10:59:31,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-20 10:59:31,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-20 10:59:31,272 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021872671700152592); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03992417054791989, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021872671700152592); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012833333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04591666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-20 10:59:31,273 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-20 10:59:31,273 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.015972379202116926, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021872671700152592); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03992417054791989, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021872671700152592); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012833333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04591666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-20 10:59:31,509 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 349 ms to try 5000 different iterations.  Found a solution that moves 1004 regions; Going from a computed imbalance of 0.015972379202116926 to a new imbalance of 7.275958166666812E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006766922918967363); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.014502602421363678); MoveCostFunction : (multiplier=7.0, imbalance=0.11155555555555556, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006766922918967363); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-20 10:59:31,511 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv35233419:78 , srv1535035759:80 , srv1780529305:81 , srv642351216:81 , srv1693002756:82 , srv841768964:82 , srv89458795:82 , srv107604990:83 , srv1172044630:83 , srv118799377:83 , srv1248938040:83 , srv1365536363:83 , srv1501927064:83 , srv1659369709:83 , srv1859795605:83 , srv245727256:83 , srv529906123:83 , srv1205914381:84 , srv1382311883:84 , srv1518986398:84 , srv172990379:84 , srv1939039804:84 , srv1986312053:84 , srv1988340987:84 , srv370149171:84 , srv39372642:84 , srv547576155:84 , srv566788738:84 , srv764446805:84 , srv787725898:84 , srv845860632:84 , srv1113817638:85 , srv1237071483:85 , srv123865504:85 , srv1538697825:85 , srv158119382:85 , srv1590947448:85 , srv1635504608:85 , srv1761706814:85 , srv192565657:85 , srv2125154937:85 , srv438375813:85 , srv524189971:85 , srv62115494:85 , srv650957900:85 , srv710906040:85 , srv784163228:85 , srv968113524:85 , srv1055799241:86 , srv1158012633:86 , srv1191705799:86 , srv1274351127:86 , srv1437552599:86 , srv1611603252:86 , srv1704743515:86 , srv1926750248:86 , srv1938328033:86 , srv1968069250:86 , srv2053248024:86 , srv218840427:86 , srv228018928:86 , srv473744443:86 , srv5481193:86 , srv623392411:86 , srv739948855:86 , srv747391714:86 , srv83162835:86 , srv1089714351:87 , srv1345565549:87 , srv1366038401:87 , srv1387273818:87 , srv1480391018:87 , srv1615442631:87 , srv2001065939:87 , srv209128286:87 , srv223323027:87 , srv30063425:87 , srv337620324:87 , srv395322939:87 , srv417160130:87 , srv461527941:87 , srv468106232:87 , srv486434451:87 , srv556050740:87 , srv887776494:87 , srv937653839:87 , srv1405171448:88 , srv1829901522:88 , srv1893634580:88 , srv1919808824:88 , srv200222919:88 , srv2039670153:88 , srv237575550:88 , srv407022649:88 , srv574203871:88 , srv711687758:88 , srv753318800:88 , srv339037555:89 , srv924581990:89 , srv13822424:550 }
2023-03-20 10:59:31,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-20 10:59:31,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv30063425=59, srv2039670153=50, srv245727256=58, srv764446805=89, srv1538697825=25, srv529906123=74, srv1387273818=18, srv2001065939=48, srv62115494=80, srv1893634580=39, srv1693002756=32, srv1437552599=20, srv753318800=88, srv566788738=78, srv417160130=67, srv1365536363=14, srv473744443=71, srv642351216=82, srv1113817638=3, srv1859795605=38, srv123865504=10, srv887776494=95, srv1704743515=33, srv710906040=84, srv711687758=85, srv747391714=87, srv1480391018=21, srv784163228=90, srv39372642=64, srv223323027=55, srv1988340987=47, srv924581990=97, srv1659369709=31, srv35233419=62, srv1405171448=19, srv2053248024=51, srv1191705799=7, srv2125154937=53, srv200222919=49, srv937653839=98, srv1366038401=15, srv486434451=72, srv1535035759=24, srv118799377=6, srv407022649=66, srv1248938040=11, srv228018928=56, srv468106232=70, srv556050740=77, srv1237071483=9, srv1055799241=0, srv5481193=76, srv370149171=63, srv1382311883=17, srv1501927064=22, srv841768964=93, srv218840427=54, srv339037555=61, srv1926750248=42, srv172990379=34, srv1919808824=40, srv1345565549=13, srv237575550=57, srv1611603252=28, srv1968069250=45, srv192565657=41, srv1089714351=2, srv787725898=91, srv574203871=79, srv968113524=99, srv1205914381=8, srv209128286=52, srv337620324=60, srv1939039804=44, srv83162835=92, srv524189971=73, srv845860632=94, srv1274351127=12, srv1829901522=37, srv1986312053=46, srv395322939=65, srv1590947448=27, srv1780529305=36, srv739948855=86, srv13822424=16, srv158119382=26, srv461527941=69, srv89458795=96, srv1761706814=35, srv107604990=1, srv623392411=81, srv1158012633=4, srv1172044630=5, srv438375813=68, srv1635504608=30, srv547576155=75, srv1938328033=43, srv1615442631=29, srv650957900=83, srv1518986398=23} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-20 10:59:31,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-20 10:59:31,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-20 10:59:31,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-20 10:59:31,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-20 10:59:31,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-20 10:59:31,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-20 10:59:31,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-20 10:59:31,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-20 10:59:31,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-20 10:59:31,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-20 10:59:31,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-20 10:59:31,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-20 10:59:31,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-20 10:59:31,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-20 10:59:31,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-20 10:59:31,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-20 10:59:31,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-20 10:59:31,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-20 10:59:31,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-20 10:59:31,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-20 10:59:31,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-20 10:59:31,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-20 10:59:31,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-20 10:59:31,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-20 10:59:31,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-20 10:59:31,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-20 10:59:31,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-20 10:59:31,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-20 10:59:31,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-20 10:59:31,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-20 10:59:31,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-20 10:59:31,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-20 10:59:31,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-20 10:59:31,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-20 10:59:31,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-20 10:59:31,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-20 10:59:31,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-20 10:59:31,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-20 10:59:31,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-20 10:59:31,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-20 10:59:31,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-20 10:59:31,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-20 10:59:31,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-20 10:59:31,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-20 10:59:31,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-20 10:59:31,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-20 10:59:31,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-20 10:59:31,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-20 10:59:31,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-20 10:59:31,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-20 10:59:31,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-20 10:59:31,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-20 10:59:31,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-20 10:59:31,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-20 10:59:31,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-20 10:59:31,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-20 10:59:31,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-20 10:59:31,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-20 10:59:31,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-20 10:59:31,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-20 10:59:31,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-20 10:59:31,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-20 10:59:31,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-20 10:59:31,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-20 10:59:31,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-20 10:59:31,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-20 10:59:31,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-20 10:59:31,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-20 10:59:31,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-20 10:59:31,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-20 10:59:31,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-20 10:59:31,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-20 10:59:31,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-20 10:59:31,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-20 10:59:31,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-20 10:59:31,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-20 10:59:31,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-20 10:59:31,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-20 10:59:31,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-20 10:59:31,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-20 10:59:31,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-20 10:59:31,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-20 10:59:31,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-20 10:59:31,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-20 10:59:31,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-20 10:59:31,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-20 10:59:31,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-20 10:59:31,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-20 10:59:31,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-20 10:59:31,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-20 10:59:31,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-20 10:59:31,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-20 10:59:31,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-20 10:59:31,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-20 10:59:31,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-20 10:59:31,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-20 10:59:31,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-20 10:59:31,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-20 10:59:31,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-20 10:59:31,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-20 10:59:31,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-20 10:59:31,663 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006766922918967363); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.014502602421363678); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006766922918967363); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-20 10:59:31,663 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-20 10:59:31,664 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=7.205656584080958E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006766922918967363); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.014502602421363678); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006766922918967363); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-20 10:59:31,998 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 442 ms to try 5000 different iterations.  Found a solution that moves 281 regions; Going from a computed imbalance of 7.205656584080958E-4 to a new imbalance of 3.711255146025602E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.470732932414788E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0022222222222222222); MoveCostFunction : (multiplier=7.0, imbalance=0.03122222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.470732932414788E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-20 10:59:31,999 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1939039804:83 , srv337620324:83 , srv486434451:83 , srv556050740:83 , srv62115494:83 , srv937653839:83 , srv1055799241:84 , srv1089714351:84 , srv1191705799:84 , srv1248938040:84 , srv1345565549:84 , srv1366038401:84 , srv1382311883:84 , srv1405171448:84 , srv1437552599:84 , srv1538697825:84 , srv1615442631:84 , srv172990379:84 , srv1919808824:84 , srv1926750248:84 , srv1968069250:84 , srv200222919:84 , srv2039670153:84 , srv218840427:84 , srv228018928:84 , srv237575550:84 , srv245727256:84 , srv30063425:84 , srv339037555:84 , srv370149171:84 , srv417160130:84 , srv438375813:84 , srv468106232:84 , srv524189971:84 , srv529906123:84 , srv5481193:84 , srv566788738:84 , srv574203871:84 , srv642351216:84 , srv784163228:84 , srv83162835:84 , srv845860632:84 , srv89458795:84 , srv924581990:84 , srv1113817638:85 , srv1158012633:85 , srv1172044630:85 , srv1205914381:85 , srv1237071483:85 , srv123865504:85 , srv1274351127:85 , srv1387273818:85 , srv1480391018:85 , srv1501927064:85 , srv1518986398:85 , srv1590947448:85 , srv1611603252:85 , srv1635504608:85 , srv1693002756:85 , srv1704743515:85 , srv1761706814:85 , srv1780529305:85 , srv1829901522:85 , srv1893634580:85 , srv192565657:85 , srv1938328033:85 , srv1986312053:85 , srv1988340987:85 , srv2001065939:85 , srv2053248024:85 , srv209128286:85 , srv2125154937:85 , srv223323027:85 , srv35233419:85 , srv39372642:85 , srv395322939:85 , srv407022649:85 , srv461527941:85 , srv473744443:85 , srv650957900:85 , srv710906040:85 , srv711687758:85 , srv747391714:85 , srv753318800:85 , srv764446805:85 , srv787725898:85 , srv887776494:85 , srv968113524:85 , srv107604990:86 , srv118799377:86 , srv1365536363:86 , srv1535035759:86 , srv158119382:86 , srv1659369709:86 , srv1859795605:86 , srv547576155:86 , srv623392411:86 , srv739948855:86 , srv841768964:86 , srv13822424:624 }
2023-03-20 10:59:32,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-20 10:59:32,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv30063425=59, srv2039670153=50, srv245727256=58, srv764446805=89, srv1538697825=25, srv529906123=74, srv1387273818=18, srv2001065939=48, srv62115494=80, srv1893634580=39, srv1693002756=32, srv1437552599=20, srv753318800=88, srv566788738=78, srv417160130=67, srv1365536363=14, srv473744443=71, srv642351216=82, srv1113817638=3, srv1859795605=38, srv123865504=10, srv887776494=95, srv1704743515=33, srv710906040=84, srv711687758=85, srv747391714=87, srv1480391018=21, srv784163228=90, srv39372642=64, srv223323027=55, srv1988340987=47, srv924581990=97, srv1659369709=31, srv35233419=62, srv1405171448=19, srv2053248024=51, srv1191705799=7, srv2125154937=53, srv200222919=49, srv937653839=98, srv1366038401=15, srv486434451=72, srv1535035759=24, srv118799377=6, srv407022649=66, srv1248938040=11, srv228018928=56, srv468106232=70, srv556050740=77, srv1237071483=9, srv1055799241=0, srv5481193=76, srv370149171=63, srv1382311883=17, srv1501927064=22, srv841768964=93, srv218840427=54, srv339037555=61, srv1926750248=42, srv172990379=34, srv1919808824=40, srv1345565549=13, srv237575550=57, srv1611603252=28, srv1968069250=45, srv192565657=41, srv1089714351=2, srv787725898=91, srv574203871=79, srv968113524=99, srv1205914381=8, srv209128286=52, srv337620324=60, srv1939039804=44, srv83162835=92, srv524189971=73, srv845860632=94, srv1274351127=12, srv1829901522=37, srv1986312053=46, srv395322939=65, srv1590947448=27, srv1780529305=36, srv739948855=86, srv13822424=16, srv158119382=26, srv461527941=69, srv89458795=96, srv1761706814=35, srv107604990=1, srv623392411=81, srv1158012633=4, srv1172044630=5, srv438375813=68, srv1635504608=30, srv547576155=75, srv1938328033=43, srv1615442631=29, srv650957900=83, srv1518986398=23} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-20 10:59:32,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-20 10:59:32,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-20 10:59:32,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-20 10:59:32,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-20 10:59:32,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-20 10:59:32,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-20 10:59:32,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-20 10:59:32,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-20 10:59:32,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-20 10:59:32,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-20 10:59:32,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-20 10:59:32,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-20 10:59:32,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-20 10:59:32,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-20 10:59:32,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-20 10:59:32,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-20 10:59:32,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-20 10:59:32,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-20 10:59:32,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-20 10:59:32,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-20 10:59:32,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-20 10:59:32,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-20 10:59:32,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-20 10:59:32,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-20 10:59:32,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-20 10:59:32,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-20 10:59:32,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-20 10:59:32,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-20 10:59:32,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-20 10:59:32,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-20 10:59:32,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-20 10:59:32,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-20 10:59:32,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-20 10:59:32,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-20 10:59:32,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-20 10:59:32,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-20 10:59:32,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-20 10:59:32,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-20 10:59:32,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-20 10:59:32,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-20 10:59:32,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-20 10:59:32,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-20 10:59:32,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-20 10:59:32,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-20 10:59:32,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-20 10:59:32,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-20 10:59:32,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-20 10:59:32,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-20 10:59:32,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-20 10:59:32,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-20 10:59:32,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-20 10:59:32,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-20 10:59:32,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-20 10:59:32,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-20 10:59:32,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-20 10:59:32,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-20 10:59:32,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-20 10:59:32,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-20 10:59:32,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-20 10:59:32,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-20 10:59:32,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-20 10:59:32,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-20 10:59:32,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-20 10:59:32,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-20 10:59:32,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-20 10:59:32,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-20 10:59:32,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-20 10:59:32,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-20 10:59:32,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-20 10:59:32,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-20 10:59:32,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-20 10:59:32,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-20 10:59:32,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-20 10:59:32,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-20 10:59:32,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-20 10:59:32,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-20 10:59:32,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-20 10:59:32,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-20 10:59:32,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-20 10:59:32,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-20 10:59:32,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-20 10:59:32,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-20 10:59:32,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-20 10:59:32,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-20 10:59:32,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-20 10:59:32,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-20 10:59:32,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-20 10:59:32,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-20 10:59:32,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-20 10:59:32,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-20 10:59:32,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-20 10:59:32,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-20 10:59:32,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-20 10:59:32,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-20 10:59:32,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-20 10:59:32,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-20 10:59:32,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-20 10:59:32,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-20 10:59:32,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-20 10:59:32,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-20 10:59:32,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-20 10:59:32,141 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.470732932414788E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0022222222222222222); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.470732932414788E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-20 10:59:32,141 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-20 10:59:32,141 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=3.51449474058113E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.470732932414788E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0022222222222222222); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.470732932414788E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-20 10:59:32,371 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 331 ms to try 5000 different iterations.  Found a solution that moves 37 regions; Going from a computed imbalance of 3.51449474058113E-5 to a new imbalance of 7.646837037730546E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0016412198797244363); MoveCostFunction : (multiplier=7.0, imbalance=0.004111111111111111); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-20 10:59:32,372 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv107604990:83 , srv1365536363:83 , srv158119382:83 , srv1659369709:83 , srv1704743515:83 , srv1829901522:83 , srv1859795605:83 , srv1893634580:83 , srv2125154937:83 , srv623392411:83 , srv739948855:83 , srv753318800:83 , srv1089714351:84 , srv1113817638:84 , srv1158012633:84 , srv1172044630:84 , srv118799377:84 , srv1191705799:84 , srv1205914381:84 , srv1237071483:84 , srv123865504:84 , srv1248938040:84 , srv1274351127:84 , srv1345565549:84 , srv1382311883:84 , srv1387273818:84 , srv1405171448:84 , srv1480391018:84 , srv1501927064:84 , srv1518986398:84 , srv1535035759:84 , srv1538697825:84 , srv1590947448:84 , srv1611603252:84 , srv1615442631:84 , srv1635504608:84 , srv1693002756:84 , srv1761706814:84 , srv1780529305:84 , srv1919808824:84 , srv192565657:84 , srv1938328033:84 , srv1968069250:84 , srv1986312053:84 , srv1988340987:84 , srv2001065939:84 , srv200222919:84 , srv2053248024:84 , srv209128286:84 , srv218840427:84 , srv223323027:84 , srv228018928:84 , srv237575550:84 , srv245727256:84 , srv30063425:84 , srv339037555:84 , srv35233419:84 , srv370149171:84 , srv39372642:84 , srv395322939:84 , srv407022649:84 , srv417160130:84 , srv438375813:84 , srv461527941:84 , srv468106232:84 , srv473744443:84 , srv524189971:84 , srv529906123:84 , srv547576155:84 , srv556050740:84 , srv566788738:84 , srv574203871:84 , srv642351216:84 , srv650957900:84 , srv710906040:84 , srv711687758:84 , srv747391714:84 , srv764446805:84 , srv784163228:84 , srv787725898:84 , srv83162835:84 , srv841768964:84 , srv845860632:84 , srv887776494:84 , srv924581990:84 , srv937653839:84 , srv968113524:84 , srv1055799241:85 , srv1366038401:85 , srv1437552599:85 , srv172990379:85 , srv1926750248:85 , srv1939039804:85 , srv2039670153:85 , srv337620324:85 , srv486434451:85 , srv5481193:85 , srv62115494:85 , srv89458795:85 , srv13822424:684 }
2023-03-20 10:59:32,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-20 10:59:32,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv30063425=59, srv2039670153=50, srv245727256=58, srv764446805=89, srv1538697825=25, srv529906123=74, srv1387273818=18, srv2001065939=48, srv62115494=80, srv1893634580=39, srv1693002756=32, srv1437552599=20, srv753318800=88, srv566788738=78, srv417160130=67, srv1365536363=14, srv473744443=71, srv642351216=82, srv1113817638=3, srv1859795605=38, srv123865504=10, srv887776494=95, srv1704743515=33, srv710906040=84, srv711687758=85, srv747391714=87, srv1480391018=21, srv784163228=90, srv39372642=64, srv223323027=55, srv1988340987=47, srv924581990=97, srv1659369709=31, srv35233419=62, srv1405171448=19, srv2053248024=51, srv1191705799=7, srv2125154937=53, srv200222919=49, srv937653839=98, srv1366038401=15, srv486434451=72, srv1535035759=24, srv118799377=6, srv407022649=66, srv1248938040=11, srv228018928=56, srv468106232=70, srv556050740=77, srv1237071483=9, srv1055799241=0, srv5481193=76, srv370149171=63, srv1382311883=17, srv1501927064=22, srv841768964=93, srv218840427=54, srv339037555=61, srv1926750248=42, srv172990379=34, srv1919808824=40, srv1345565549=13, srv237575550=57, srv1611603252=28, srv1968069250=45, srv192565657=41, srv1089714351=2, srv787725898=91, srv574203871=79, srv968113524=99, srv1205914381=8, srv209128286=52, srv337620324=60, srv1939039804=44, srv83162835=92, srv524189971=73, srv845860632=94, srv1274351127=12, srv1829901522=37, srv1986312053=46, srv395322939=65, srv1590947448=27, srv1780529305=36, srv739948855=86, srv13822424=16, srv158119382=26, srv461527941=69, srv89458795=96, srv1761706814=35, srv107604990=1, srv623392411=81, srv1158012633=4, srv1172044630=5, srv438375813=68, srv1635504608=30, srv547576155=75, srv1938328033=43, srv1615442631=29, srv650957900=83, srv1518986398=23} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-20 10:59:32,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-20 10:59:32,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-20 10:59:32,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-20 10:59:32,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-20 10:59:32,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-20 10:59:32,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-20 10:59:32,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-20 10:59:32,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-20 10:59:32,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-20 10:59:32,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-20 10:59:32,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-20 10:59:32,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-20 10:59:32,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-20 10:59:32,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-20 10:59:32,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-20 10:59:32,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-20 10:59:32,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-20 10:59:32,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-20 10:59:32,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-20 10:59:32,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-20 10:59:32,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-20 10:59:32,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-20 10:59:32,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-20 10:59:32,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-20 10:59:32,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-20 10:59:32,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-20 10:59:32,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-20 10:59:32,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-20 10:59:32,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-20 10:59:32,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-20 10:59:32,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-20 10:59:32,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-20 10:59:32,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-20 10:59:32,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-20 10:59:32,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-20 10:59:32,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-20 10:59:32,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-20 10:59:32,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-20 10:59:32,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-20 10:59:32,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-20 10:59:32,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-20 10:59:32,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-20 10:59:32,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-20 10:59:32,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-20 10:59:32,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-20 10:59:32,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-20 10:59:32,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-20 10:59:32,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-20 10:59:32,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-20 10:59:32,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-20 10:59:32,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-20 10:59:32,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-20 10:59:32,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-20 10:59:32,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-20 10:59:32,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-20 10:59:32,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-20 10:59:32,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-20 10:59:32,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-20 10:59:32,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-20 10:59:32,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-20 10:59:32,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-20 10:59:32,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-20 10:59:32,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-20 10:59:32,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-20 10:59:32,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-20 10:59:32,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-20 10:59:32,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-20 10:59:32,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-20 10:59:32,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-20 10:59:32,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-20 10:59:32,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-20 10:59:32,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-20 10:59:32,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-20 10:59:32,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-20 10:59:32,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-20 10:59:32,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-20 10:59:32,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-20 10:59:32,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-20 10:59:32,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-20 10:59:32,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-20 10:59:32,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-20 10:59:32,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-20 10:59:32,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-20 10:59:32,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-20 10:59:32,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-20 10:59:32,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-20 10:59:32,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-20 10:59:32,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-20 10:59:32,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-20 10:59:32,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-20 10:59:32,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-20 10:59:32,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-20 10:59:32,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-20 10:59:32,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-20 10:59:32,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-20 10:59:32,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-20 10:59:32,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-20 10:59:32,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-20 10:59:32,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-20 10:59:32,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-20 10:59:32,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-20 10:59:32,527 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=7.387757500312559E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0016412198797244363); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-20 10:59:32,527 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv107604990:83 , srv1365536363:83 , srv158119382:83 , srv1659369709:83 , srv1704743515:83 , srv1829901522:83 , srv1859795605:83 , srv1893634580:83 , srv2125154937:83 , srv623392411:83 , srv739948855:83 , srv753318800:83 , srv1089714351:84 , srv1113817638:84 , srv1158012633:84 , srv1172044630:84 , srv118799377:84 , srv1191705799:84 , srv1205914381:84 , srv1237071483:84 , srv123865504:84 , srv1248938040:84 , srv1274351127:84 , srv1345565549:84 , srv1382311883:84 , srv1387273818:84 , srv1405171448:84 , srv1480391018:84 , srv1501927064:84 , srv1518986398:84 , srv1535035759:84 , srv1538697825:84 , srv1590947448:84 , srv1611603252:84 , srv1615442631:84 , srv1635504608:84 , srv1693002756:84 , srv1761706814:84 , srv1780529305:84 , srv1919808824:84 , srv192565657:84 , srv1938328033:84 , srv1968069250:84 , srv1986312053:84 , srv1988340987:84 , srv2001065939:84 , srv200222919:84 , srv2053248024:84 , srv209128286:84 , srv218840427:84 , srv223323027:84 , srv228018928:84 , srv237575550:84 , srv245727256:84 , srv30063425:84 , srv339037555:84 , srv35233419:84 , srv370149171:84 , srv39372642:84 , srv395322939:84 , srv407022649:84 , srv417160130:84 , srv438375813:84 , srv461527941:84 , srv468106232:84 , srv473744443:84 , srv524189971:84 , srv529906123:84 , srv547576155:84 , srv556050740:84 , srv566788738:84 , srv574203871:84 , srv642351216:84 , srv650957900:84 , srv710906040:84 , srv711687758:84 , srv747391714:84 , srv764446805:84 , srv784163228:84 , srv787725898:84 , srv83162835:84 , srv841768964:84 , srv845860632:84 , srv887776494:84 , srv924581990:84 , srv937653839:84 , srv968113524:84 , srv1055799241:85 , srv1366038401:85 , srv1437552599:85 , srv172990379:85 , srv1926750248:85 , srv1939039804:85 , srv2039670153:85 , srv337620324:85 , srv486434451:85 , srv5481193:85 , srv62115494:85 , srv89458795:85 , srv13822424:684 }
2023-03-20 10:59:32,571 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=780 (was 744) - SystemLoadAverage LEAK? -, ProcessCount=183 (was 183), AvailableMemoryMB=10392 (was 11168)
2023-03-20 10:59:32,589 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=780, ProcessCount=183, AvailableMemoryMB=10391
2023-03-20 10:59:32,589 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-20 10:59:32,590 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-03-20 10:59:32,603 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1066318947:84 , srv1149285048:84 , srv1163534212:84 , srv1182673812:84 , srv1230270903:84 , srv1235757684:84 , srv1355271389:84 , srv1430385930:84 , srv1529559703:84 , srv1568245402:84 , srv1651063919:84 , srv1690083937:84 , srv1736261974:84 , srv2001599716:84 , srv2089430367:84 , srv2146755189:84 , srv302142492:84 , srv37671776:84 , srv48961482:84 , srv578081254:84 , srv621705634:84 , srv656770421:84 , srv751883603:84 , srv806840694:84 , srv816567279:84 , srv861672850:84 , srv891667063:84 , srv94451833:84 , srv977784345:84 , srv1607266092:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-03-20 10:59:32,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-20 10:59:32,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1235757684=5, srv816567279=25, srv1230270903=4, srv1651063919=11, srv1690083937=12, srv806840694=24, srv1607266092=10, srv1149285048=1, srv2089430367=15, srv2146755189=16, srv1430385930=7, srv1182673812=3, srv1066318947=0, srv2001599716=14, srv48961482=19, srv1355271389=6, srv621705634=21, srv891667063=27, srv1163534212=2, srv861672850=26, srv1529559703=8, srv37671776=18, srv656770421=22, srv977784345=29, srv1736261974=13, srv94451833=28, srv578081254=20, srv751883603=23, srv302142492=17, srv1568245402=9} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-20 10:59:32,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-20 10:59:32,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-20 10:59:32,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-20 10:59:32,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-20 10:59:32,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-20 10:59:32,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-20 10:59:32,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-20 10:59:32,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-20 10:59:32,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-20 10:59:32,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-20 10:59:32,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-20 10:59:32,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-20 10:59:32,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-20 10:59:32,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-20 10:59:32,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-20 10:59:32,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-20 10:59:32,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-20 10:59:32,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-20 10:59:32,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-20 10:59:32,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-20 10:59:32,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-20 10:59:32,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-20 10:59:32,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-20 10:59:32,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-20 10:59:32,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-20 10:59:32,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-20 10:59:32,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-20 10:59:32,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-20 10:59:32,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-20 10:59:32,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-20 10:59:32,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-03-20 10:59:32,654 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1394795894747982, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-20 10:59:32,654 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912632538896287, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1394795894747982, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-03-20 11:01:32,630 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 7868955 different iterations.  Found a solution that moves 1852 regions; Going from a computed imbalance of 0.9912632538896287 to a new imbalance of 4.322660390073086E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6859259259259259, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-20 11:01:32,638 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1066318947:90 , srv1149285048:90 , srv1163534212:90 , srv1182673812:90 , srv1230270903:90 , srv1235757684:90 , srv1355271389:90 , srv1430385930:90 , srv1529559703:90 , srv1568245402:90 , srv1607266092:90 , srv1651063919:90 , srv1690083937:90 , srv1736261974:90 , srv2001599716:90 , srv2089430367:90 , srv2146755189:90 , srv302142492:90 , srv37671776:90 , srv48961482:90 , srv578081254:90 , srv621705634:90 , srv656770421:90 , srv751883603:90 , srv806840694:90 , srv816567279:90 , srv861672850:90 , srv891667063:90 , srv94451833:90 , srv977784345:90 }
2023-03-20 11:01:32,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-20 11:01:32,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1235757684=5, srv816567279=25, srv1230270903=4, srv1651063919=11, srv1690083937=12, srv806840694=24, srv1607266092=10, srv1149285048=1, srv2089430367=15, srv2146755189=16, srv1430385930=7, srv1182673812=3, srv1066318947=0, srv2001599716=14, srv48961482=19, srv1355271389=6, srv621705634=21, srv891667063=27, srv1163534212=2, srv861672850=26, srv1529559703=8, srv37671776=18, srv656770421=22, srv977784345=29, srv1736261974=13, srv94451833=28, srv578081254=20, srv751883603=23, srv302142492=17, srv1568245402=9} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-20 11:01:32,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-20 11:01:32,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-20 11:01:32,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-20 11:01:32,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-20 11:01:32,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-20 11:01:32,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-20 11:01:32,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-20 11:01:32,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-20 11:01:32,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-20 11:01:32,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-20 11:01:32,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-20 11:01:32,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-20 11:01:32,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-20 11:01:32,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-20 11:01:32,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-20 11:01:32,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-20 11:01:32,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-20 11:01:32,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-20 11:01:32,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-20 11:01:32,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-20 11:01:32,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-20 11:01:32,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-20 11:01:32,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-20 11:01:32,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-20 11:01:32,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-20 11:01:32,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-20 11:01:32,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-20 11:01:32,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-20 11:01:32,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-20 11:01:32,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-20 11:01:32,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-03-20 11:01:32,658 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-20 11:01:32,658 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1066318947:90 , srv1149285048:90 , srv1163534212:90 , srv1182673812:90 , srv1230270903:90 , srv1235757684:90 , srv1355271389:90 , srv1430385930:90 , srv1529559703:90 , srv1568245402:90 , srv1607266092:90 , srv1651063919:90 , srv1690083937:90 , srv1736261974:90 , srv2001599716:90 , srv2089430367:90 , srv2146755189:90 , srv302142492:90 , srv37671776:90 , srv48961482:90 , srv578081254:90 , srv621705634:90 , srv656770421:90 , srv751883603:90 , srv806840694:90 , srv816567279:90 , srv861672850:90 , srv891667063:90 , srv94451833:90 , srv977784345:90 }
2023-03-20 11:01:32,675 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=595 (was 780), ProcessCount=180 (was 183), AvailableMemoryMB=10695 (was 10391) - AvailableMemoryMB LEAK? -