2023-08-09 22:53:11,826 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-08-09 22:53:12,182 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-08-09 22:53:12,303 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-08-09 22:53:12,369 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-09 22:53:12,379 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-08-09 22:53:12,397 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=214, ProcessCount=171, AvailableMemoryMB=10979
2023-08-09 22:53:12,399 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-09 22:53:12,400 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-08-09 22:53:12,546 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv101852454:84 , srv1024206572:84 , srv1060873665:84 , srv1069939227:84 , srv1078909391:84 , srv1098891847:84 , srv1119747760:84 , srv1180154249:84 , srv1184501769:84 , srv1190835667:84 , srv1216560234:84 , srv12871362:84 , srv1302274031:84 , srv1305402891:84 , srv1342628480:84 , srv1346104122:84 , srv1360336496:84 , srv1361807364:84 , srv1373794671:84 , srv1389205922:84 , srv139843249:84 , srv140209280:84 , srv1403967957:84 , srv1426484969:84 , srv1435506037:84 , srv1472360412:84 , srv1565709370:84 , srv1602541014:84 , srv1609891786:84 , srv1623220518:84 , srv1624340701:84 , srv1626502859:84 , srv1635494549:84 , srv1664217934:84 , srv166509493:84 , srv1671158963:84 , srv1710906511:84 , srv174897206:84 , srv1761524119:84 , srv1806267020:84 , srv1809912986:84 , srv1816884801:84 , srv1817814656:84 , srv1825461485:84 , srv1956110070:84 , srv1987254670:84 , srv1993605311:84 , srv1999811518:84 , srv2000241555:84 , srv2030075732:84 , srv2030995143:84 , srv2045718047:84 , srv2101614017:84 , srv2102759709:84 , srv2114014181:84 , srv2133410038:84 , srv2138584560:84 , srv2140980440:84 , srv220759193:84 , srv231405810:84 , srv239952460:84 , srv24715864:84 , srv251044032:84 , srv255722264:84 , srv283865705:84 , srv323764053:84 , srv341959157:84 , srv344412716:84 , srv38385237:84 , srv391945751:84 , srv42065149:84 , srv431543856:84 , srv442827560:84 , srv482989717:84 , srv496658974:84 , srv522122303:84 , srv592315192:84 , srv601039235:84 , srv62729368:84 , srv627359150:84 , srv630759856:84 , srv687130317:84 , srv736111022:84 , srv744539826:84 , srv748122591:84 , srv771968596:84 , srv776490713:84 , srv786635526:84 , srv788209292:84 , srv793528209:84 , srv811852796:84 , srv825481577:84 , srv871956221:84 , srv902706732:84 , srv912052255:84 , srv939803505:84 , srv973118411:84 , srv994019219:84 , srv994268204:84 , srv857098810:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-08-09 22:53:12,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-09 22:53:12,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv391945751=69, srv736111022=82, srv1987254670=45, srv1602541014=27, srv220759193=58, srv771968596=85, srv522122303=75, srv496658974=74, srv601039235=77, srv2045718047=51, srv1389205922=19, srv139843249=20, srv592315192=76, srv748122591=84, srv1024206572=1, srv1069939227=3, srv793528209=89, srv994019219=98, srv2133410038=55, srv38385237=68, srv1098891847=5, srv1999811518=47, srv140209280=21, srv231405810=59, srv776490713=86, srv1216560234=10, srv2140980440=57, srv1472360412=25, srv687130317=81, srv62729368=78, srv2000241555=48, srv101852454=0, srv630759856=80, srv174897206=37, srv1635494549=32, srv24715864=61, srv431543856=71, srv857098810=92, srv902706732=94, srv2030995143=50, srv1664217934=33, srv973118411=97, srv912052255=95, srv1993605311=46, srv1180154249=7, srv283865705=64, srv1806267020=39, srv1184501769=8, srv1817814656=42, srv2114014181=54, srv1403967957=22, srv1956110070=44, srv2030075732=49, srv1190835667=9, srv1816884801=41, srv12871362=11, srv2138584560=56, srv1373794671=18, srv1305402891=13, srv341959157=66, srv1361807364=17, srv1710906511=36, srv2101614017=52, srv1825461485=43, srv2102759709=53, srv871956221=93, srv1761524119=38, srv825481577=91, srv442827560=72, srv1671158963=35, srv786635526=87, srv166509493=34, srv788209292=88, srv1809912986=40, srv42065149=70, srv994268204=99, srv1626502859=31, srv1342628480=14, srv1119747760=6, srv939803505=96, srv255722264=63, srv1078909391=4, srv1346104122=15, srv1302274031=12, srv811852796=90, srv627359150=79, srv239952460=60, srv1624340701=30, srv1360336496=16, srv1426484969=23, srv482989717=73, srv1623220518=29, srv344412716=67, srv1060873665=2, srv1609891786=28, srv744539826=83, srv323764053=65, srv1565709370=26, srv251044032=62, srv1435506037=24} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-09 22:53:12,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-09 22:53:12,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-09 22:53:12,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-09 22:53:12,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-09 22:53:12,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-09 22:53:12,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-09 22:53:12,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-09 22:53:12,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-09 22:53:12,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-09 22:53:12,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-09 22:53:12,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-09 22:53:12,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-09 22:53:12,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-09 22:53:12,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-09 22:53:12,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-09 22:53:12,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-09 22:53:12,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-09 22:53:12,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-09 22:53:12,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-09 22:53:12,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-09 22:53:12,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-09 22:53:12,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-09 22:53:12,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-09 22:53:12,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-09 22:53:12,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-09 22:53:12,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-09 22:53:12,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-09 22:53:12,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-09 22:53:12,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-09 22:53:12,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-09 22:53:12,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-09 22:53:12,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-09 22:53:12,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-09 22:53:12,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-09 22:53:12,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-09 22:53:12,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-09 22:53:12,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-09 22:53:12,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-09 22:53:12,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-09 22:53:12,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-09 22:53:12,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-09 22:53:12,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-09 22:53:12,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-09 22:53:12,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-09 22:53:12,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-09 22:53:12,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-09 22:53:12,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-09 22:53:12,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-09 22:53:12,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-09 22:53:12,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-09 22:53:12,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-09 22:53:12,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-09 22:53:12,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-09 22:53:12,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-09 22:53:12,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-09 22:53:12,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-09 22:53:12,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-09 22:53:12,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-09 22:53:12,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-09 22:53:12,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-09 22:53:12,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-09 22:53:12,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-09 22:53:12,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-09 22:53:12,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-09 22:53:12,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-09 22:53:12,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-09 22:53:12,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-09 22:53:12,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-09 22:53:12,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-09 22:53:12,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-09 22:53:12,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-09 22:53:12,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-09 22:53:12,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-09 22:53:12,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-09 22:53:12,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-09 22:53:12,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-09 22:53:12,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-09 22:53:12,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-09 22:53:12,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-09 22:53:12,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-09 22:53:12,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-09 22:53:12,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-09 22:53:12,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-09 22:53:12,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-09 22:53:12,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-09 22:53:12,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-09 22:53:12,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-09 22:53:12,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-09 22:53:12,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-09 22:53:12,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-09 22:53:12,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-09 22:53:12,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-09 22:53:12,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-09 22:53:12,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-09 22:53:12,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-09 22:53:12,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-09 22:53:12,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-09 22:53:12,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-09 22:53:12,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-09 22:53:12,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-09 22:53:12,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-09 22:53:12,856 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-09 22:53:12,856 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-09 22:53:12,857 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-09 22:53:13,252 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 572 ms to try 5000 different iterations.  Found a solution that moves 4685 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18168551535895092. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03970680675345763, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.057389918048405444, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5205555555555555, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03970680675345763, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1765, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24775, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-09 22:53:13,287 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv231405810:80 , srv251044032:80 , srv431543856:80 , srv1373794671:81 , srv140209280:81 , srv687130317:81 , srv736111022:81 , srv1342628480:82 , srv1565709370:82 , srv174897206:82 , srv744539826:82 , srv825481577:82 , srv912052255:82 , srv1360336496:83 , srv1426484969:83 , srv1435506037:83 , srv1624340701:83 , srv1999811518:83 , srv2030995143:83 , srv2133410038:83 , srv42065149:83 , srv811852796:83 , srv1305402891:84 , srv1389205922:84 , srv1609891786:84 , srv1635494549:84 , srv1664217934:84 , srv1817814656:84 , srv1993605311:84 , srv592315192:84 , srv601039235:84 , srv1060873665:85 , srv1119747760:85 , srv1180154249:85 , srv1671158963:85 , srv2102759709:85 , srv38385237:85 , srv482989717:85 , srv496658974:85 , srv62729368:85 , srv627359150:85 , srv771968596:85 , srv902706732:85 , srv1078909391:86 , srv1190835667:86 , srv1403967957:86 , srv1806267020:86 , srv1956110070:86 , srv344412716:86 , srv793528209:86 , srv871956221:86 , srv994019219:86 , srv994268204:86 , srv1069939227:87 , srv1472360412:87 , srv166509493:87 , srv1825461485:87 , srv2114014181:87 , srv239952460:87 , srv255722264:87 , srv748122591:87 , srv788209292:87 , srv1098891847:88 , srv12871362:88 , srv1623220518:88 , srv1626502859:88 , srv1816884801:88 , srv2138584560:88 , srv24715864:88 , srv341959157:88 , srv391945751:88 , srv522122303:88 , srv786635526:88 , srv973118411:88 , srv101852454:89 , srv1184501769:89 , srv1216560234:89 , srv1302274031:89 , srv2000241555:89 , srv2045718047:89 , srv220759193:89 , srv630759856:89 , srv1761524119:90 , srv2030075732:90 , srv776490713:90 , srv1361807364:91 , srv1602541014:91 , srv1809912986:91 , srv323764053:91 , srv939803505:91 , srv1024206572:92 , srv139843249:92 , srv2101614017:92 , srv2140980440:92 , srv283865705:93 , srv1987254670:94 , srv442827560:95 , srv1346104122:96 , srv1710906511:97 , srv857098810:442 }
2023-08-09 22:53:13,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-09 22:53:13,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv391945751=69, srv736111022=82, srv1987254670=45, srv1602541014=27, srv220759193=58, srv771968596=85, srv522122303=75, srv496658974=74, srv601039235=77, srv2045718047=51, srv1389205922=19, srv139843249=20, srv592315192=76, srv748122591=84, srv1024206572=1, srv1069939227=3, srv793528209=89, srv994019219=98, srv2133410038=55, srv38385237=68, srv1098891847=5, srv1999811518=47, srv140209280=21, srv231405810=59, srv776490713=86, srv1216560234=10, srv2140980440=57, srv1472360412=25, srv687130317=81, srv62729368=78, srv2000241555=48, srv101852454=0, srv630759856=80, srv174897206=37, srv1635494549=32, srv24715864=61, srv431543856=71, srv857098810=92, srv902706732=94, srv2030995143=50, srv1664217934=33, srv973118411=97, srv912052255=95, srv1993605311=46, srv1180154249=7, srv283865705=64, srv1806267020=39, srv1184501769=8, srv1817814656=42, srv2114014181=54, srv1403967957=22, srv1956110070=44, srv2030075732=49, srv1190835667=9, srv1816884801=41, srv12871362=11, srv2138584560=56, srv1373794671=18, srv1305402891=13, srv341959157=66, srv1361807364=17, srv1710906511=36, srv2101614017=52, srv1825461485=43, srv2102759709=53, srv871956221=93, srv1761524119=38, srv825481577=91, srv442827560=72, srv1671158963=35, srv786635526=87, srv166509493=34, srv788209292=88, srv1809912986=40, srv42065149=70, srv994268204=99, srv1626502859=31, srv1342628480=14, srv1119747760=6, srv939803505=96, srv255722264=63, srv1078909391=4, srv1346104122=15, srv1302274031=12, srv811852796=90, srv627359150=79, srv239952460=60, srv1624340701=30, srv1360336496=16, srv1426484969=23, srv482989717=73, srv1623220518=29, srv344412716=67, srv1060873665=2, srv1609891786=28, srv744539826=83, srv323764053=65, srv1565709370=26, srv251044032=62, srv1435506037=24} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-09 22:53:13,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-09 22:53:13,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-09 22:53:13,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-09 22:53:13,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-09 22:53:13,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-09 22:53:13,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-09 22:53:13,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-09 22:53:13,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-09 22:53:13,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-09 22:53:13,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-09 22:53:13,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-09 22:53:13,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-09 22:53:13,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-09 22:53:13,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-09 22:53:13,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-09 22:53:13,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-09 22:53:13,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-09 22:53:13,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-09 22:53:13,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-09 22:53:13,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-09 22:53:13,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-09 22:53:13,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-09 22:53:13,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-09 22:53:13,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-09 22:53:13,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-09 22:53:13,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-09 22:53:13,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-09 22:53:13,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-09 22:53:13,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-09 22:53:13,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-09 22:53:13,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-09 22:53:13,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-09 22:53:13,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-09 22:53:13,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-09 22:53:13,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-09 22:53:13,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-09 22:53:13,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-09 22:53:13,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-09 22:53:13,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-09 22:53:13,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-09 22:53:13,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-09 22:53:13,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-09 22:53:13,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-09 22:53:13,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-09 22:53:13,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-09 22:53:13,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-09 22:53:13,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-09 22:53:13,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-09 22:53:13,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-09 22:53:13,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-09 22:53:13,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-09 22:53:13,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-09 22:53:13,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-09 22:53:13,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-09 22:53:13,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-09 22:53:13,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-09 22:53:13,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-09 22:53:13,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-09 22:53:13,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-09 22:53:13,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-09 22:53:13,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-09 22:53:13,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-09 22:53:13,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-09 22:53:13,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-09 22:53:13,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-09 22:53:13,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-09 22:53:13,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-09 22:53:13,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-09 22:53:13,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-09 22:53:13,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-09 22:53:13,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-09 22:53:13,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-09 22:53:13,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-09 22:53:13,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-09 22:53:13,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-09 22:53:13,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-09 22:53:13,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-09 22:53:13,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-09 22:53:13,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-09 22:53:13,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-09 22:53:13,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-09 22:53:13,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-09 22:53:13,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-09 22:53:13,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-09 22:53:13,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-09 22:53:13,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-09 22:53:13,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-09 22:53:13,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-09 22:53:13,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-09 22:53:13,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-09 22:53:13,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-09 22:53:13,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-09 22:53:13,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-09 22:53:13,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-09 22:53:13,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-09 22:53:13,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-09 22:53:13,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-09 22:53:13,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-09 22:53:13,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-09 22:53:13,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-09 22:53:13,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-09 22:53:13,523 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03970680675345763, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.057389918048405444, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03970680675345763, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1765, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24775, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-09 22:53:13,523 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-09 22:53:13,524 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.1816527102877941, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03970680675345763, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.057389918048405444, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03970680675345763, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1765, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24775, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-09 22:53:13,866 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 496 ms to try 5000 different iterations.  Found a solution that moves 2696 regions; Going from a computed imbalance of 0.1816527102877941 to a new imbalance of 0.014735461485358847. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02372684056006958); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.032293546974689995, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2995555555555556, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02372684056006958); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01175); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043083333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-09 22:53:13,888 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1346104122:76 , srv1710906511:76 , srv442827560:77 , srv1987254670:78 , srv1024206572:79 , srv139843249:79 , srv283865705:79 , srv2101614017:80 , srv776490713:80 , srv323764053:81 , srv1361807364:82 , srv1602541014:82 , srv1809912986:82 , srv2030075732:82 , srv391945751:82 , srv630759856:82 , srv939803505:82 , srv1216560234:83 , srv12871362:83 , srv1626502859:83 , srv1761524119:83 , srv2045718047:83 , srv2140980440:83 , srv220759193:83 , srv522122303:83 , srv786635526:83 , srv101852454:84 , srv1078909391:84 , srv1098891847:84 , srv1184501769:84 , srv1302274031:84 , srv1472360412:84 , srv1623220518:84 , srv166509493:84 , srv2000241555:84 , srv24715864:84 , srv255722264:84 , srv482989717:84 , srv771968596:84 , srv973118411:84 , srv1069939227:85 , srv1190835667:85 , srv1671158963:85 , srv1825461485:85 , srv2114014181:85 , srv2138584560:85 , srv341959157:85 , srv788209292:85 , srv1060873665:86 , srv1389205922:86 , srv1806267020:86 , srv1816884801:86 , srv1817814656:86 , srv1956110070:86 , srv2102759709:86 , srv496658974:86 , srv592315192:86 , srv627359150:86 , srv748122591:86 , srv994019219:86 , srv994268204:86 , srv1119747760:87 , srv1360336496:87 , srv1403967957:87 , srv1609891786:87 , srv1635494549:87 , srv1664217934:87 , srv239952460:87 , srv38385237:87 , srv601039235:87 , srv62729368:87 , srv1305402891:88 , srv1342628480:88 , srv1435506037:88 , srv1565709370:88 , srv174897206:88 , srv344412716:88 , srv744539826:88 , srv871956221:88 , srv1180154249:89 , srv1624340701:89 , srv1993605311:89 , srv1999811518:89 , srv42065149:89 , srv431543856:89 , srv793528209:89 , srv902706732:89 , srv1373794671:90 , srv140209280:90 , srv1426484969:90 , srv687130317:90 , srv912052255:90 , srv2030995143:91 , srv2133410038:91 , srv251044032:91 , srv811852796:91 , srv736111022:92 , srv825481577:92 , srv231405810:93 , srv857098810:543 }
2023-08-09 22:53:13,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-09 22:53:13,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv391945751=69, srv736111022=82, srv1987254670=45, srv1602541014=27, srv220759193=58, srv771968596=85, srv522122303=75, srv496658974=74, srv601039235=77, srv2045718047=51, srv1389205922=19, srv139843249=20, srv592315192=76, srv748122591=84, srv1024206572=1, srv1069939227=3, srv793528209=89, srv994019219=98, srv2133410038=55, srv38385237=68, srv1098891847=5, srv1999811518=47, srv140209280=21, srv231405810=59, srv776490713=86, srv1216560234=10, srv2140980440=57, srv1472360412=25, srv687130317=81, srv62729368=78, srv2000241555=48, srv101852454=0, srv630759856=80, srv174897206=37, srv1635494549=32, srv24715864=61, srv431543856=71, srv857098810=92, srv902706732=94, srv2030995143=50, srv1664217934=33, srv973118411=97, srv912052255=95, srv1993605311=46, srv1180154249=7, srv283865705=64, srv1806267020=39, srv1184501769=8, srv1817814656=42, srv2114014181=54, srv1403967957=22, srv1956110070=44, srv2030075732=49, srv1190835667=9, srv1816884801=41, srv12871362=11, srv2138584560=56, srv1373794671=18, srv1305402891=13, srv341959157=66, srv1361807364=17, srv1710906511=36, srv2101614017=52, srv1825461485=43, srv2102759709=53, srv871956221=93, srv1761524119=38, srv825481577=91, srv442827560=72, srv1671158963=35, srv786635526=87, srv166509493=34, srv788209292=88, srv1809912986=40, srv42065149=70, srv994268204=99, srv1626502859=31, srv1342628480=14, srv1119747760=6, srv939803505=96, srv255722264=63, srv1078909391=4, srv1346104122=15, srv1302274031=12, srv811852796=90, srv627359150=79, srv239952460=60, srv1624340701=30, srv1360336496=16, srv1426484969=23, srv482989717=73, srv1623220518=29, srv344412716=67, srv1060873665=2, srv1609891786=28, srv744539826=83, srv323764053=65, srv1565709370=26, srv251044032=62, srv1435506037=24} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-09 22:53:13,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-09 22:53:13,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-09 22:53:13,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-09 22:53:13,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-09 22:53:13,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-09 22:53:13,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-09 22:53:13,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-09 22:53:13,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-09 22:53:13,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-09 22:53:13,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-09 22:53:13,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-09 22:53:13,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-09 22:53:13,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-09 22:53:13,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-09 22:53:13,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-09 22:53:13,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-09 22:53:13,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-09 22:53:13,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-09 22:53:13,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-09 22:53:13,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-09 22:53:13,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-09 22:53:13,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-09 22:53:13,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-09 22:53:13,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-09 22:53:13,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-09 22:53:13,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-09 22:53:13,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-09 22:53:13,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-09 22:53:13,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-09 22:53:13,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-09 22:53:13,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-09 22:53:13,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-09 22:53:13,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-09 22:53:13,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-09 22:53:13,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-09 22:53:13,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-09 22:53:13,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-09 22:53:13,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-09 22:53:13,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-09 22:53:13,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-09 22:53:13,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-09 22:53:13,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-09 22:53:13,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-09 22:53:13,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-09 22:53:13,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-09 22:53:13,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-09 22:53:13,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-09 22:53:13,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-09 22:53:13,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-09 22:53:13,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-09 22:53:13,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-09 22:53:13,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-09 22:53:13,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-09 22:53:13,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-09 22:53:13,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-09 22:53:13,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-09 22:53:13,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-09 22:53:13,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-09 22:53:13,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-09 22:53:13,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-09 22:53:13,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-09 22:53:13,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-09 22:53:13,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-09 22:53:13,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-09 22:53:13,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-09 22:53:13,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-09 22:53:13,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-09 22:53:13,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-09 22:53:13,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-09 22:53:13,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-09 22:53:13,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-09 22:53:13,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-09 22:53:13,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-09 22:53:13,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-09 22:53:13,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-09 22:53:13,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-09 22:53:13,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-09 22:53:13,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-09 22:53:13,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-09 22:53:13,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-09 22:53:13,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-09 22:53:13,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-09 22:53:13,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-09 22:53:13,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-09 22:53:13,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-09 22:53:13,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-09 22:53:13,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-09 22:53:13,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-09 22:53:13,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-09 22:53:13,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-09 22:53:13,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-09 22:53:13,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-09 22:53:13,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-09 22:53:13,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-09 22:53:13,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-09 22:53:13,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-09 22:53:13,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-09 22:53:13,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-09 22:53:13,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-09 22:53:13,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-09 22:53:13,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-09 22:53:14,100 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02372684056006958); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.032293546974689995, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02372684056006958); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01175); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043083333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-09 22:53:14,101 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-09 22:53:14,101 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.01471658368987563, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02372684056006958); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.032293546974689995, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02372684056006958); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01175); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043083333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-09 22:53:14,390 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 412 ms to try 5000 different iterations.  Found a solution that moves 1011 regions; Going from a computed imbalance of 0.01471658368987563 to a new imbalance of 5.356292848345879E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005988724492745198); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006011586269238994); MoveCostFunction : (multiplier=7.0, imbalance=0.11233333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005988724492745198); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-09 22:53:14,393 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv793528209:82 , srv1180154249:83 , srv2030995143:83 , srv2133410038:83 , srv2140980440:83 , srv239952460:83 , srv344412716:83 , srv811852796:83 , srv101852454:84 , srv1361807364:84 , srv1403967957:84 , srv1710906511:84 , srv1761524119:84 , srv1993605311:84 , srv2000241555:84 , srv231405810:84 , srv341959157:84 , srv736111022:84 , srv825481577:84 , srv902706732:84 , srv939803505:84 , srv1119747760:85 , srv1184501769:85 , srv1216560234:85 , srv1302274031:85 , srv1305402891:85 , srv1426484969:85 , srv1602541014:85 , srv1623220518:85 , srv1624340701:85 , srv1809912986:85 , srv1816884801:85 , srv1956110070:85 , srv2045718047:85 , srv2101614017:85 , srv2114014181:85 , srv2138584560:85 , srv220759193:85 , srv24715864:85 , srv283865705:85 , srv323764053:85 , srv38385237:85 , srv442827560:85 , srv788209292:85 , srv871956221:85 , srv973118411:85 , srv1069939227:86 , srv1098891847:86 , srv12871362:86 , srv1346104122:86 , srv1373794671:86 , srv1435506037:86 , srv1626502859:86 , srv1635494549:86 , srv1664217934:86 , srv166509493:86 , srv1806267020:86 , srv1987254670:86 , srv1999811518:86 , srv2030075732:86 , srv2102759709:86 , srv251044032:86 , srv42065149:86 , srv496658974:86 , srv522122303:86 , srv62729368:86 , srv627359150:86 , srv630759856:86 , srv748122591:86 , srv912052255:86 , srv994019219:86 , srv994268204:86 , srv1024206572:87 , srv1060873665:87 , srv1078909391:87 , srv1190835667:87 , srv1342628480:87 , srv1360336496:87 , srv1389205922:87 , srv139843249:87 , srv140209280:87 , srv1472360412:87 , srv1609891786:87 , srv1825461485:87 , srv255722264:87 , srv391945751:87 , srv482989717:87 , srv592315192:87 , srv687130317:87 , srv744539826:87 , srv786635526:87 , srv1671158963:88 , srv174897206:88 , srv1817814656:88 , srv431543856:88 , srv601039235:88 , srv771968596:88 , srv776490713:88 , srv1565709370:89 , srv857098810:526 }
2023-08-09 22:53:14,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-09 22:53:14,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv391945751=69, srv736111022=82, srv1987254670=45, srv1602541014=27, srv220759193=58, srv771968596=85, srv522122303=75, srv496658974=74, srv601039235=77, srv2045718047=51, srv1389205922=19, srv139843249=20, srv592315192=76, srv748122591=84, srv1024206572=1, srv1069939227=3, srv793528209=89, srv994019219=98, srv2133410038=55, srv38385237=68, srv1098891847=5, srv1999811518=47, srv140209280=21, srv231405810=59, srv776490713=86, srv1216560234=10, srv2140980440=57, srv1472360412=25, srv687130317=81, srv62729368=78, srv2000241555=48, srv101852454=0, srv630759856=80, srv174897206=37, srv1635494549=32, srv24715864=61, srv431543856=71, srv857098810=92, srv902706732=94, srv2030995143=50, srv1664217934=33, srv973118411=97, srv912052255=95, srv1993605311=46, srv1180154249=7, srv283865705=64, srv1806267020=39, srv1184501769=8, srv1817814656=42, srv2114014181=54, srv1403967957=22, srv1956110070=44, srv2030075732=49, srv1190835667=9, srv1816884801=41, srv12871362=11, srv2138584560=56, srv1373794671=18, srv1305402891=13, srv341959157=66, srv1361807364=17, srv1710906511=36, srv2101614017=52, srv1825461485=43, srv2102759709=53, srv871956221=93, srv1761524119=38, srv825481577=91, srv442827560=72, srv1671158963=35, srv786635526=87, srv166509493=34, srv788209292=88, srv1809912986=40, srv42065149=70, srv994268204=99, srv1626502859=31, srv1342628480=14, srv1119747760=6, srv939803505=96, srv255722264=63, srv1078909391=4, srv1346104122=15, srv1302274031=12, srv811852796=90, srv627359150=79, srv239952460=60, srv1624340701=30, srv1360336496=16, srv1426484969=23, srv482989717=73, srv1623220518=29, srv344412716=67, srv1060873665=2, srv1609891786=28, srv744539826=83, srv323764053=65, srv1565709370=26, srv251044032=62, srv1435506037=24} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-09 22:53:14,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-09 22:53:14,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-09 22:53:14,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-09 22:53:14,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-09 22:53:14,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-09 22:53:14,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-09 22:53:14,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-09 22:53:14,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-09 22:53:14,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-09 22:53:14,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-09 22:53:14,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-09 22:53:14,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-09 22:53:14,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-09 22:53:14,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-09 22:53:14,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-09 22:53:14,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-09 22:53:14,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-09 22:53:14,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-09 22:53:14,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-09 22:53:14,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-09 22:53:14,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-09 22:53:14,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-09 22:53:14,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-09 22:53:14,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-09 22:53:14,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-09 22:53:14,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-09 22:53:14,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-09 22:53:14,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-09 22:53:14,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-09 22:53:14,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-09 22:53:14,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-09 22:53:14,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-09 22:53:14,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-09 22:53:14,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-09 22:53:14,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-09 22:53:14,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-09 22:53:14,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-09 22:53:14,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-09 22:53:14,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-09 22:53:14,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-09 22:53:14,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-09 22:53:14,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-09 22:53:14,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-09 22:53:14,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-09 22:53:14,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-09 22:53:14,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-09 22:53:14,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-09 22:53:14,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-09 22:53:14,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-09 22:53:14,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-09 22:53:14,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-09 22:53:14,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-09 22:53:14,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-09 22:53:14,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-09 22:53:14,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-09 22:53:14,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-09 22:53:14,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-09 22:53:14,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-09 22:53:14,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-09 22:53:14,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-09 22:53:14,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-09 22:53:14,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-09 22:53:14,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-09 22:53:14,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-09 22:53:14,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-09 22:53:14,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-09 22:53:14,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-09 22:53:14,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-09 22:53:14,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-09 22:53:14,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-09 22:53:14,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-09 22:53:14,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-09 22:53:14,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-09 22:53:14,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-09 22:53:14,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-09 22:53:14,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-09 22:53:14,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-09 22:53:14,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-09 22:53:14,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-09 22:53:14,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-09 22:53:14,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-09 22:53:14,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-09 22:53:14,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-09 22:53:14,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-09 22:53:14,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-09 22:53:14,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-09 22:53:14,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-09 22:53:14,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-09 22:53:14,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-09 22:53:14,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-09 22:53:14,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-09 22:53:14,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-09 22:53:14,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-09 22:53:14,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-09 22:53:14,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-09 22:53:14,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-09 22:53:14,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-09 22:53:14,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-09 22:53:14,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-09 22:53:14,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-09 22:53:14,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-09 22:53:14,549 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005988724492745198); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006011586269238994); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005988724492745198); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-09 22:53:14,550 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-09 22:53:14,550 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=5.285501115283829E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005988724492745198); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006011586269238994); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005988724492745198); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-09 22:53:14,915 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 472 ms to try 5000 different iterations.  Found a solution that moves 278 regions; Going from a computed imbalance of 5.285501115283829E-4 to a new imbalance of 1.3145172473983734E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0019534423844256277); MoveCostFunction : (multiplier=7.0, imbalance=0.03088888888888889, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-09 22:53:14,917 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1565709370:83 , srv1825461485:83 , srv748122591:83 , srv101852454:84 , srv1024206572:84 , srv1060873665:84 , srv1069939227:84 , srv1098891847:84 , srv1119747760:84 , srv1184501769:84 , srv1190835667:84 , srv1302274031:84 , srv1346104122:84 , srv1360336496:84 , srv1361807364:84 , srv1373794671:84 , srv139843249:84 , srv140209280:84 , srv1426484969:84 , srv1435506037:84 , srv1472360412:84 , srv1602541014:84 , srv1609891786:84 , srv1624340701:84 , srv1635494549:84 , srv1806267020:84 , srv1809912986:84 , srv1816884801:84 , srv1817814656:84 , srv1987254670:84 , srv2030075732:84 , srv2030995143:84 , srv2138584560:84 , srv2140980440:84 , srv255722264:84 , srv42065149:84 , srv601039235:84 , srv62729368:84 , srv630759856:84 , srv687130317:84 , srv776490713:84 , srv786635526:84 , srv825481577:84 , srv871956221:84 , srv902706732:84 , srv912052255:84 , srv994019219:84 , srv994268204:84 , srv1078909391:85 , srv1180154249:85 , srv1216560234:85 , srv12871362:85 , srv1305402891:85 , srv1342628480:85 , srv1389205922:85 , srv1403967957:85 , srv1623220518:85 , srv1626502859:85 , srv1664217934:85 , srv1671158963:85 , srv1710906511:85 , srv174897206:85 , srv1956110070:85 , srv1993605311:85 , srv1999811518:85 , srv2000241555:85 , srv2045718047:85 , srv2101614017:85 , srv2102759709:85 , srv2114014181:85 , srv220759193:85 , srv231405810:85 , srv239952460:85 , srv24715864:85 , srv251044032:85 , srv283865705:85 , srv323764053:85 , srv341959157:85 , srv344412716:85 , srv38385237:85 , srv391945751:85 , srv431543856:85 , srv442827560:85 , srv496658974:85 , srv522122303:85 , srv592315192:85 , srv627359150:85 , srv736111022:85 , srv744539826:85 , srv771968596:85 , srv788209292:85 , srv793528209:85 , srv811852796:85 , srv973118411:85 , srv166509493:86 , srv1761524119:86 , srv2133410038:86 , srv482989717:86 , srv939803505:86 , srv857098810:631 }
2023-08-09 22:53:14,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-09 22:53:14,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv391945751=69, srv736111022=82, srv1987254670=45, srv1602541014=27, srv220759193=58, srv771968596=85, srv522122303=75, srv496658974=74, srv601039235=77, srv2045718047=51, srv1389205922=19, srv139843249=20, srv592315192=76, srv748122591=84, srv1024206572=1, srv1069939227=3, srv793528209=89, srv994019219=98, srv2133410038=55, srv38385237=68, srv1098891847=5, srv1999811518=47, srv140209280=21, srv231405810=59, srv776490713=86, srv1216560234=10, srv2140980440=57, srv1472360412=25, srv687130317=81, srv62729368=78, srv2000241555=48, srv101852454=0, srv630759856=80, srv174897206=37, srv1635494549=32, srv24715864=61, srv431543856=71, srv857098810=92, srv902706732=94, srv2030995143=50, srv1664217934=33, srv973118411=97, srv912052255=95, srv1993605311=46, srv1180154249=7, srv283865705=64, srv1806267020=39, srv1184501769=8, srv1817814656=42, srv2114014181=54, srv1403967957=22, srv1956110070=44, srv2030075732=49, srv1190835667=9, srv1816884801=41, srv12871362=11, srv2138584560=56, srv1373794671=18, srv1305402891=13, srv341959157=66, srv1361807364=17, srv1710906511=36, srv2101614017=52, srv1825461485=43, srv2102759709=53, srv871956221=93, srv1761524119=38, srv825481577=91, srv442827560=72, srv1671158963=35, srv786635526=87, srv166509493=34, srv788209292=88, srv1809912986=40, srv42065149=70, srv994268204=99, srv1626502859=31, srv1342628480=14, srv1119747760=6, srv939803505=96, srv255722264=63, srv1078909391=4, srv1346104122=15, srv1302274031=12, srv811852796=90, srv627359150=79, srv239952460=60, srv1624340701=30, srv1360336496=16, srv1426484969=23, srv482989717=73, srv1623220518=29, srv344412716=67, srv1060873665=2, srv1609891786=28, srv744539826=83, srv323764053=65, srv1565709370=26, srv251044032=62, srv1435506037=24} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-09 22:53:14,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-09 22:53:14,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-09 22:53:14,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-09 22:53:14,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-09 22:53:14,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-09 22:53:14,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-09 22:53:14,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-09 22:53:14,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-09 22:53:14,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-09 22:53:14,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-09 22:53:14,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-09 22:53:14,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-09 22:53:14,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-09 22:53:14,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-09 22:53:14,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-09 22:53:14,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-09 22:53:14,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-09 22:53:14,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-09 22:53:14,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-09 22:53:14,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-09 22:53:14,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-09 22:53:14,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-09 22:53:14,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-09 22:53:14,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-09 22:53:14,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-09 22:53:14,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-09 22:53:14,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-09 22:53:14,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-09 22:53:14,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-09 22:53:14,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-09 22:53:14,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-09 22:53:14,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-09 22:53:14,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-09 22:53:14,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-09 22:53:14,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-09 22:53:14,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-09 22:53:14,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-09 22:53:14,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-09 22:53:14,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-09 22:53:14,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-09 22:53:14,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-09 22:53:14,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-09 22:53:14,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-09 22:53:14,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-09 22:53:14,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-09 22:53:14,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-09 22:53:14,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-09 22:53:14,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-09 22:53:14,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-09 22:53:14,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-09 22:53:14,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-09 22:53:14,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-09 22:53:14,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-09 22:53:14,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-09 22:53:14,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-09 22:53:14,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-09 22:53:14,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-09 22:53:14,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-09 22:53:14,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-09 22:53:14,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-09 22:53:14,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-09 22:53:14,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-09 22:53:14,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-09 22:53:14,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-09 22:53:14,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-09 22:53:14,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-09 22:53:14,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-09 22:53:14,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-09 22:53:14,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-09 22:53:14,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-09 22:53:14,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-09 22:53:14,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-09 22:53:14,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-09 22:53:14,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-09 22:53:14,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-09 22:53:14,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-09 22:53:14,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-09 22:53:14,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-09 22:53:14,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-09 22:53:14,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-09 22:53:14,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-09 22:53:14,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-09 22:53:14,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-09 22:53:14,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-09 22:53:14,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-09 22:53:14,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-09 22:53:14,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-09 22:53:14,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-09 22:53:14,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-09 22:53:14,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-09 22:53:14,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-09 22:53:14,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-09 22:53:14,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-09 22:53:14,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-09 22:53:14,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-09 22:53:14,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-09 22:53:14,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-09 22:53:14,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-09 22:53:14,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-09 22:53:14,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-09 22:53:14,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-09 22:53:15,103 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.1198574868518857E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0019534423844256277); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-09 22:53:15,103 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1565709370:83 , srv1825461485:83 , srv748122591:83 , srv101852454:84 , srv1024206572:84 , srv1060873665:84 , srv1069939227:84 , srv1098891847:84 , srv1119747760:84 , srv1184501769:84 , srv1190835667:84 , srv1302274031:84 , srv1346104122:84 , srv1360336496:84 , srv1361807364:84 , srv1373794671:84 , srv139843249:84 , srv140209280:84 , srv1426484969:84 , srv1435506037:84 , srv1472360412:84 , srv1602541014:84 , srv1609891786:84 , srv1624340701:84 , srv1635494549:84 , srv1806267020:84 , srv1809912986:84 , srv1816884801:84 , srv1817814656:84 , srv1987254670:84 , srv2030075732:84 , srv2030995143:84 , srv2138584560:84 , srv2140980440:84 , srv255722264:84 , srv42065149:84 , srv601039235:84 , srv62729368:84 , srv630759856:84 , srv687130317:84 , srv776490713:84 , srv786635526:84 , srv825481577:84 , srv871956221:84 , srv902706732:84 , srv912052255:84 , srv994019219:84 , srv994268204:84 , srv1078909391:85 , srv1180154249:85 , srv1216560234:85 , srv12871362:85 , srv1305402891:85 , srv1342628480:85 , srv1389205922:85 , srv1403967957:85 , srv1623220518:85 , srv1626502859:85 , srv1664217934:85 , srv1671158963:85 , srv1710906511:85 , srv174897206:85 , srv1956110070:85 , srv1993605311:85 , srv1999811518:85 , srv2000241555:85 , srv2045718047:85 , srv2101614017:85 , srv2102759709:85 , srv2114014181:85 , srv220759193:85 , srv231405810:85 , srv239952460:85 , srv24715864:85 , srv251044032:85 , srv283865705:85 , srv323764053:85 , srv341959157:85 , srv344412716:85 , srv38385237:85 , srv391945751:85 , srv431543856:85 , srv442827560:85 , srv496658974:85 , srv522122303:85 , srv592315192:85 , srv627359150:85 , srv736111022:85 , srv744539826:85 , srv771968596:85 , srv788209292:85 , srv793528209:85 , srv811852796:85 , srv973118411:85 , srv166509493:86 , srv1761524119:86 , srv2133410038:86 , srv482989717:86 , srv939803505:86 , srv857098810:631 }
2023-08-09 22:53:15,138 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=214 (was 214), ProcessCount=171 (was 171), AvailableMemoryMB=10198 (was 10979)
2023-08-09 22:53:15,149 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=214, ProcessCount=171, AvailableMemoryMB=10199
2023-08-09 22:53:15,150 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-09 22:53:15,151 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-08-09 22:53:15,162 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1023129692:84 , srv1023685137:84 , srv1029516681:84 , srv1046927765:84 , srv1118430524:84 , srv1252214101:84 , srv1311366335:84 , srv1497075446:84 , srv1539357568:84 , srv1597364276:84 , srv1630803082:84 , srv1645099574:84 , srv1654859176:84 , srv165937785:84 , srv1739729181:84 , srv2017145754:84 , srv265622186:84 , srv298477019:84 , srv313272662:84 , srv446831715:84 , srv470247313:84 , srv536436541:84 , srv624861138:84 , srv659558535:84 , srv736751636:84 , srv795920366:84 , srv852891385:84 , srv952303057:84 , srv975556182:84 , srv9485592:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-08-09 22:53:15,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-09 22:53:15,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv795920366=25, srv1539357568=8, srv536436541=21, srv298477019=17, srv659558535=23, srv852891385=26, srv313272662=18, srv446831715=19, srv470247313=20, srv2017145754=15, srv952303057=28, srv736751636=24, srv1739729181=14, srv265622186=16, srv1497075446=7, srv1046927765=3, srv1118430524=4, srv1023685137=1, srv624861138=22, srv1311366335=6, srv975556182=29, srv9485592=27, srv1029516681=2, srv1645099574=11, srv1252214101=5, srv1597364276=9, srv1023129692=0, srv1654859176=12, srv1630803082=10, srv165937785=13} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-09 22:53:15,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-09 22:53:15,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-09 22:53:15,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-09 22:53:15,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-09 22:53:15,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-09 22:53:15,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-09 22:53:15,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-09 22:53:15,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-09 22:53:15,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-09 22:53:15,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-09 22:53:15,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-09 22:53:15,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-09 22:53:15,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-09 22:53:15,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-09 22:53:15,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-09 22:53:15,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-09 22:53:15,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-09 22:53:15,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-09 22:53:15,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-09 22:53:15,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-09 22:53:15,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-09 22:53:15,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-09 22:53:15,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-09 22:53:15,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-09 22:53:15,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-09 22:53:15,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-09 22:53:15,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-09 22:53:15,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-09 22:53:15,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-09 22:53:15,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-09 22:53:15,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-08-09 22:53:15,208 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.110331715363071, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-09 22:53:15,208 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912540694896727, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.110331715363071, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-08-09 22:55:15,186 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6250776 different iterations.  Found a solution that moves 1850 regions; Going from a computed imbalance of 0.9912540694896727 to a new imbalance of 4.317992290299789E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6851851851851852, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-09 22:55:15,192 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1023129692:90 , srv1023685137:90 , srv1029516681:90 , srv1046927765:90 , srv1118430524:90 , srv1252214101:90 , srv1311366335:90 , srv1497075446:90 , srv1539357568:90 , srv1597364276:90 , srv1630803082:90 , srv1645099574:90 , srv1654859176:90 , srv165937785:90 , srv1739729181:90 , srv2017145754:90 , srv265622186:90 , srv298477019:90 , srv313272662:90 , srv446831715:90 , srv470247313:90 , srv536436541:90 , srv624861138:90 , srv659558535:90 , srv736751636:90 , srv795920366:90 , srv852891385:90 , srv9485592:90 , srv952303057:90 , srv975556182:90 }
2023-08-09 22:55:15,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-09 22:55:15,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv795920366=25, srv1539357568=8, srv536436541=21, srv298477019=17, srv659558535=23, srv852891385=26, srv313272662=18, srv446831715=19, srv470247313=20, srv2017145754=15, srv952303057=28, srv736751636=24, srv1739729181=14, srv265622186=16, srv1497075446=7, srv1046927765=3, srv1118430524=4, srv1023685137=1, srv624861138=22, srv1311366335=6, srv975556182=29, srv9485592=27, srv1029516681=2, srv1645099574=11, srv1252214101=5, srv1597364276=9, srv1023129692=0, srv1654859176=12, srv1630803082=10, srv165937785=13} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-09 22:55:15,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-09 22:55:15,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-09 22:55:15,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-09 22:55:15,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-09 22:55:15,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-09 22:55:15,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-09 22:55:15,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-09 22:55:15,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-09 22:55:15,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-09 22:55:15,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-09 22:55:15,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-09 22:55:15,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-09 22:55:15,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-09 22:55:15,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-09 22:55:15,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-09 22:55:15,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-09 22:55:15,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-09 22:55:15,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-09 22:55:15,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-09 22:55:15,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-09 22:55:15,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-09 22:55:15,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-09 22:55:15,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-09 22:55:15,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-09 22:55:15,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-09 22:55:15,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-09 22:55:15,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-09 22:55:15,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-09 22:55:15,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-09 22:55:15,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-09 22:55:15,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-08-09 22:55:15,211 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-09 22:55:15,211 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1023129692:90 , srv1023685137:90 , srv1029516681:90 , srv1046927765:90 , srv1118430524:90 , srv1252214101:90 , srv1311366335:90 , srv1497075446:90 , srv1539357568:90 , srv1597364276:90 , srv1630803082:90 , srv1645099574:90 , srv1654859176:90 , srv165937785:90 , srv1739729181:90 , srv2017145754:90 , srv265622186:90 , srv298477019:90 , srv313272662:90 , srv446831715:90 , srv470247313:90 , srv536436541:90 , srv624861138:90 , srv659558535:90 , srv736751636:90 , srv795920366:90 , srv852891385:90 , srv9485592:90 , srv952303057:90 , srv975556182:90 }
2023-08-09 22:55:15,228 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=173 (was 214), ProcessCount=171 (was 171), AvailableMemoryMB=10056 (was 10199)