2023-04-13 20:55:39,188 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-04-13 20:55:39,499 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-04-13 20:55:39,618 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-04-13 20:55:39,678 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-13 20:55:39,686 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-04-13 20:55:39,701 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=393, ProcessCount=173, AvailableMemoryMB=1329
2023-04-13 20:55:39,703 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-13 20:55:39,703 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-04-13 20:55:39,833 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1016132481:84 , srv1033539982:84 , srv1042193282:84 , srv1043872124:84 , srv1066502924:84 , srv106847771:84 , srv1088629657:84 , srv1118780806:84 , srv1131420607:84 , srv1143398149:84 , srv1173086086:84 , srv1187671331:84 , srv1200246990:84 , srv1202303595:84 , srv1202627306:84 , srv1234480838:84 , srv1236285584:84 , srv1275747967:84 , srv129668858:84 , srv130901814:84 , srv1330928572:84 , srv1412359723:84 , srv1425962419:84 , srv1444374977:84 , srv1468297931:84 , srv1471718915:84 , srv1499646716:84 , srv1505990858:84 , srv1511434724:84 , srv1518201249:84 , srv1522688193:84 , srv1541467116:84 , srv1549720844:84 , srv1563542205:84 , srv1613993269:84 , srv1635844033:84 , srv1692584678:84 , srv1704076044:84 , srv1728799952:84 , srv1738780175:84 , srv1743987270:84 , srv1747873967:84 , srv175260503:84 , srv1770066928:84 , srv1784956813:84 , srv1790854501:84 , srv179386267:84 , srv1802673845:84 , srv1830503925:84 , srv184899678:84 , srv1851110860:84 , srv1868731164:84 , srv1920122272:84 , srv1923500409:84 , srv1943916076:84 , srv1972584978:84 , srv197629290:84 , srv197677319:84 , srv1979523027:84 , srv1992954394:84 , srv1996861352:84 , srv2025563547:84 , srv2039808084:84 , srv205462963:84 , srv2063301711:84 , srv2067919109:84 , srv2098929128:84 , srv243315215:84 , srv25383225:84 , srv272476360:84 , srv275640169:84 , srv279782949:84 , srv28072949:84 , srv28291732:84 , srv315987812:84 , srv37926455:84 , srv426588467:84 , srv491190656:84 , srv537017713:84 , srv549211949:84 , srv552184188:84 , srv589633638:84 , srv592965112:84 , srv595147186:84 , srv598151573:84 , srv619746174:84 , srv661509789:84 , srv675907970:84 , srv692413303:84 , srv757152295:84 , srv773201815:84 , srv791289425:84 , srv79782957:84 , srv824504377:84 , srv860877796:84 , srv889050897:84 , srv900389265:84 , srv971062617:84 , srv984734483:84 , srv517780356:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-04-13 20:55:39,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-13 20:55:39,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1471718915=25, srv860877796=95, srv1499646716=26, srv28072949=72, srv25383225=68, srv1511434724=28, srv1802673845=47, srv1743987270=40, srv1330928572=20, srv1541467116=31, srv1692584678=36, srv757152295=90, srv1033539982=1, srv2067919109=65, srv184899678=49, srv595147186=84, srv79782957=93, srv130901814=19, srv517780356=78, srv1131420607=8, srv2025563547=61, srv1549720844=32, srv1143398149=9, srv1728799952=38, srv197629290=56, srv692413303=89, srv1043872124=3, srv1868731164=51, srv1738780175=39, srv2098929128=66, srv619746174=86, srv28291732=73, srv279782949=71, srv1770066928=43, srv243315215=67, srv1088629657=6, srv1202303595=13, srv1066502924=4, srv175260503=42, srv197677319=57, srv1202627306=14, srv1412359723=21, srv549211949=80, srv129668858=18, srv1704076044=37, srv1920122272=52, srv1505990858=27, srv1943916076=54, srv1275747967=17, srv1444374977=23, srv589633638=82, srv1992954394=59, srv1830503925=48, srv1851110860=50, srv1118780806=7, srv1784956813=44, srv1173086086=10, srv205462963=63, srv179386267=46, srv106847771=5, srv1425962419=22, srv537017713=79, srv675907970=88, srv1923500409=53, srv773201815=91, srv2063301711=64, srv275640169=70, srv1234480838=15, srv2039808084=62, srv37926455=75, srv1747873967=41, srv1042193282=2, srv900389265=97, srv971062617=98, srv592965112=83, srv1200246990=12, srv426588467=76, srv1187671331=11, srv1522688193=30, srv552184188=81, srv1613993269=34, srv1979523027=58, srv1468297931=24, srv1563542205=33, srv272476360=69, srv1236285584=16, srv1972584978=55, srv598151573=85, srv1016132481=0, srv889050897=96, srv491190656=77, srv791289425=92, srv1790854501=45, srv984734483=99, srv1518201249=29, srv1996861352=60, srv315987812=74, srv661509789=87, srv1635844033=35, srv824504377=94} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-13 20:55:39,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-13 20:55:39,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-13 20:55:39,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-13 20:55:39,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-13 20:55:39,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-13 20:55:39,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-13 20:55:39,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-13 20:55:39,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-13 20:55:39,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-13 20:55:39,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-13 20:55:39,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-13 20:55:39,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-13 20:55:39,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-13 20:55:39,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-13 20:55:39,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-13 20:55:39,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-13 20:55:39,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-13 20:55:39,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-13 20:55:39,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-13 20:55:39,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-13 20:55:39,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-13 20:55:39,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-13 20:55:39,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-13 20:55:39,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-13 20:55:39,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-13 20:55:39,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-13 20:55:39,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-13 20:55:39,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-13 20:55:39,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-13 20:55:39,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-13 20:55:39,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-13 20:55:39,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-13 20:55:39,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-13 20:55:39,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-13 20:55:39,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-13 20:55:39,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-13 20:55:39,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-13 20:55:39,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-13 20:55:39,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-13 20:55:39,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-13 20:55:39,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-13 20:55:39,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-13 20:55:39,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-13 20:55:39,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-13 20:55:39,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-13 20:55:39,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-13 20:55:39,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-13 20:55:39,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-13 20:55:39,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-13 20:55:39,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-13 20:55:39,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-13 20:55:39,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-13 20:55:39,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-13 20:55:39,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-13 20:55:39,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-13 20:55:39,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-13 20:55:39,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-13 20:55:39,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-13 20:55:39,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-13 20:55:39,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-13 20:55:39,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-13 20:55:39,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-13 20:55:39,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-13 20:55:39,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-13 20:55:39,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-13 20:55:39,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-13 20:55:39,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-13 20:55:39,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-13 20:55:39,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-13 20:55:39,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-13 20:55:39,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-13 20:55:39,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-13 20:55:39,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-13 20:55:39,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-13 20:55:39,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-13 20:55:39,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-13 20:55:39,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-13 20:55:39,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-13 20:55:39,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-13 20:55:39,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-13 20:55:39,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-13 20:55:39,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-13 20:55:39,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-13 20:55:39,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-13 20:55:39,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-13 20:55:39,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-13 20:55:39,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-13 20:55:39,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-13 20:55:39,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-13 20:55:39,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-13 20:55:39,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-13 20:55:39,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-13 20:55:39,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-13 20:55:39,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-13 20:55:39,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-13 20:55:39,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-13 20:55:39,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-13 20:55:39,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-13 20:55:39,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-13 20:55:39,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-13 20:55:39,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-13 20:55:40,214 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-13 20:55:40,214 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-13 20:55:40,215 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-13 20:55:40,839 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 885 ms to try 5000 different iterations.  Found a solution that moves 4721 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1750521554475711. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04125539639413686, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05871615604686937, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5245555555555556, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04125539639413686, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17025, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23641666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-13 20:55:40,861 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1234480838:83 , srv1236285584:83 , srv1275747967:83 , srv1518201249:83 , srv1770066928:83 , srv2063301711:83 , srv25383225:83 , srv773201815:83 , srv984734483:83 , srv1499646716:84 , srv1738780175:84 , srv1747873967:84 , srv1868731164:84 , srv1943916076:84 , srv1996861352:84 , srv2067919109:84 , srv275640169:84 , srv28291732:84 , srv426588467:84 , srv692413303:84 , srv889050897:84 , srv1016132481:85 , srv1042193282:85 , srv1043872124:85 , srv1143398149:85 , srv1173086086:85 , srv130901814:85 , srv1425962419:85 , srv1468297931:85 , srv1522688193:85 , srv1541467116:85 , srv175260503:85 , srv1790854501:85 , srv179386267:85 , srv272476360:85 , srv1088629657:86 , srv1118780806:86 , srv1131420607:86 , srv1412359723:86 , srv1511434724:86 , srv1563542205:86 , srv1613993269:86 , srv1728799952:86 , srv1802673845:86 , srv184899678:86 , srv1923500409:86 , srv2025563547:86 , srv243315215:86 , srv279782949:86 , srv28072949:86 , srv491190656:86 , srv537017713:86 , srv549211949:86 , srv552184188:86 , srv592965112:86 , srv598151573:86 , srv824504377:86 , srv129668858:87 , srv1330928572:87 , srv1743987270:87 , srv1784956813:87 , srv197629290:87 , srv197677319:87 , srv1992954394:87 , srv2039808084:87 , srv205462963:87 , srv2098929128:87 , srv37926455:87 , srv661509789:87 , srv757152295:87 , srv791289425:87 , srv900389265:87 , srv971062617:87 , srv1033539982:88 , srv1066502924:88 , srv106847771:88 , srv1202303595:88 , srv1202627306:88 , srv1444374977:88 , srv1505990858:88 , srv1549720844:88 , srv1692584678:88 , srv1704076044:88 , srv1979523027:88 , srv675907970:88 , srv860877796:88 , srv1187671331:89 , srv1200246990:89 , srv1635844033:89 , srv1830503925:89 , srv1920122272:89 , srv1972584978:89 , srv589633638:89 , srv1471718915:90 , srv315987812:90 , srv79782957:90 , srv1851110860:92 , srv619746174:92 , srv595147186:93 , srv517780356:457 }
2023-04-13 20:55:40,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-13 20:55:40,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1471718915=25, srv860877796=95, srv1499646716=26, srv28072949=72, srv25383225=68, srv1511434724=28, srv1802673845=47, srv1743987270=40, srv1330928572=20, srv1541467116=31, srv1692584678=36, srv757152295=90, srv1033539982=1, srv2067919109=65, srv184899678=49, srv595147186=84, srv79782957=93, srv130901814=19, srv517780356=78, srv1131420607=8, srv2025563547=61, srv1549720844=32, srv1143398149=9, srv1728799952=38, srv197629290=56, srv692413303=89, srv1043872124=3, srv1868731164=51, srv1738780175=39, srv2098929128=66, srv619746174=86, srv28291732=73, srv279782949=71, srv1770066928=43, srv243315215=67, srv1088629657=6, srv1202303595=13, srv1066502924=4, srv175260503=42, srv197677319=57, srv1202627306=14, srv1412359723=21, srv549211949=80, srv129668858=18, srv1704076044=37, srv1920122272=52, srv1505990858=27, srv1943916076=54, srv1275747967=17, srv1444374977=23, srv589633638=82, srv1992954394=59, srv1830503925=48, srv1851110860=50, srv1118780806=7, srv1784956813=44, srv1173086086=10, srv205462963=63, srv179386267=46, srv106847771=5, srv1425962419=22, srv537017713=79, srv675907970=88, srv1923500409=53, srv773201815=91, srv2063301711=64, srv275640169=70, srv1234480838=15, srv2039808084=62, srv37926455=75, srv1747873967=41, srv1042193282=2, srv900389265=97, srv971062617=98, srv592965112=83, srv1200246990=12, srv426588467=76, srv1187671331=11, srv1522688193=30, srv552184188=81, srv1613993269=34, srv1979523027=58, srv1468297931=24, srv1563542205=33, srv272476360=69, srv1236285584=16, srv1972584978=55, srv598151573=85, srv1016132481=0, srv889050897=96, srv491190656=77, srv791289425=92, srv1790854501=45, srv984734483=99, srv1518201249=29, srv1996861352=60, srv315987812=74, srv661509789=87, srv1635844033=35, srv824504377=94} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-13 20:55:40,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-13 20:55:40,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-13 20:55:40,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-13 20:55:40,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-13 20:55:40,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-13 20:55:40,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-13 20:55:40,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-13 20:55:40,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-13 20:55:40,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-13 20:55:40,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-13 20:55:40,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-13 20:55:40,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-13 20:55:40,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-13 20:55:40,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-13 20:55:40,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-13 20:55:40,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-13 20:55:40,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-13 20:55:40,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-13 20:55:40,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-13 20:55:40,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-13 20:55:40,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-13 20:55:40,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-13 20:55:40,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-13 20:55:40,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-13 20:55:40,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-13 20:55:40,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-13 20:55:40,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-13 20:55:40,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-13 20:55:40,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-13 20:55:40,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-13 20:55:40,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-13 20:55:40,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-13 20:55:40,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-13 20:55:40,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-13 20:55:40,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-13 20:55:40,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-13 20:55:40,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-13 20:55:40,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-13 20:55:40,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-13 20:55:40,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-13 20:55:40,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-13 20:55:40,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-13 20:55:40,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-13 20:55:40,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-13 20:55:40,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-13 20:55:40,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-13 20:55:40,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-13 20:55:40,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-13 20:55:40,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-13 20:55:40,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-13 20:55:40,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-13 20:55:40,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-13 20:55:40,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-13 20:55:40,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-13 20:55:40,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-13 20:55:40,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-13 20:55:40,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-13 20:55:40,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-13 20:55:40,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-13 20:55:40,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-13 20:55:40,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-13 20:55:40,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-13 20:55:40,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-13 20:55:40,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-13 20:55:40,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-13 20:55:40,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-13 20:55:40,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-13 20:55:40,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-13 20:55:40,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-13 20:55:40,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-13 20:55:40,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-13 20:55:40,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-13 20:55:40,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-13 20:55:40,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-13 20:55:40,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-13 20:55:40,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-13 20:55:40,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-13 20:55:40,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-13 20:55:40,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-13 20:55:40,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-13 20:55:40,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-13 20:55:40,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-13 20:55:40,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-13 20:55:40,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-13 20:55:40,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-13 20:55:40,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-13 20:55:40,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-13 20:55:40,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-13 20:55:40,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-13 20:55:40,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-13 20:55:40,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-13 20:55:40,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-13 20:55:40,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-13 20:55:40,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-13 20:55:40,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-13 20:55:40,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-13 20:55:40,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-13 20:55:40,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-13 20:55:40,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-13 20:55:40,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-13 20:55:40,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-13 20:55:41,060 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04125539639413686, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05871615604686937, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04125539639413686, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17025, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23641666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-13 20:55:41,060 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-13 20:55:41,060 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.1750190982990265, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04125539639413686, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05871615604686937, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04125539639413686, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17025, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23641666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-13 20:55:41,350 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 430 ms to try 5000 different iterations.  Found a solution that moves 2635 regions; Going from a computed imbalance of 0.1750190982990265 to a new imbalance of 0.013228315001625359. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022513799789134994); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03386776954115048, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2927777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022513799789134994); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010166666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.042166666666666665, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-13 20:55:41,358 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1471718915:79 , srv1635844033:80 , srv1692584678:81 , srv1704076044:81 , srv315987812:81 , srv589633638:81 , srv595147186:81 , srv1444374977:82 , srv1851110860:82 , srv197677319:82 , srv2098929128:82 , srv619746174:82 , srv675907970:82 , srv1187671331:83 , srv197629290:83 , srv79782957:83 , srv824504377:83 , srv1033539982:84 , srv1066502924:84 , srv1200246990:84 , srv1202303595:84 , srv1541467116:84 , srv1549720844:84 , srv1830503925:84 , srv184899678:84 , srv1979523027:84 , srv2039808084:84 , srv598151573:84 , srv860877796:84 , srv106847771:85 , srv1088629657:85 , srv1118780806:85 , srv1330928572:85 , srv1563542205:85 , srv1802673845:85 , srv205462963:85 , srv491190656:85 , srv537017713:85 , srv592965112:85 , srv900389265:85 , srv1043872124:86 , srv1173086086:86 , srv129668858:86 , srv130901814:86 , srv1505990858:86 , srv1511434724:86 , srv1522688193:86 , srv175260503:86 , srv1790854501:86 , srv1920122272:86 , srv1923500409:86 , srv1972584978:86 , srv1992954394:86 , srv2063301711:86 , srv28072949:86 , srv661509789:86 , srv791289425:86 , srv1016132481:87 , srv1042193282:87 , srv1131420607:87 , srv1202627306:87 , srv1234480838:87 , srv1412359723:87 , srv1425962419:87 , srv1613993269:87 , srv1728799952:87 , srv1743987270:87 , srv1784956813:87 , srv179386267:87 , srv1943916076:87 , srv2067919109:87 , srv243315215:87 , srv275640169:87 , srv37926455:87 , srv549211949:87 , srv552184188:87 , srv692413303:87 , srv757152295:87 , srv971062617:87 , srv1143398149:88 , srv1738780175:88 , srv1747873967:88 , srv1868731164:88 , srv1996861352:88 , srv2025563547:88 , srv272476360:88 , srv426588467:88 , srv773201815:88 , srv889050897:88 , srv1468297931:89 , srv1499646716:89 , srv1518201249:89 , srv1770066928:89 , srv25383225:89 , srv279782949:89 , srv1236285584:90 , srv1275747967:90 , srv28291732:90 , srv984734483:92 , srv517780356:517 }
2023-04-13 20:55:41,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-13 20:55:41,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1471718915=25, srv860877796=95, srv1499646716=26, srv28072949=72, srv25383225=68, srv1511434724=28, srv1802673845=47, srv1743987270=40, srv1330928572=20, srv1541467116=31, srv1692584678=36, srv757152295=90, srv1033539982=1, srv2067919109=65, srv184899678=49, srv595147186=84, srv79782957=93, srv130901814=19, srv517780356=78, srv1131420607=8, srv2025563547=61, srv1549720844=32, srv1143398149=9, srv1728799952=38, srv197629290=56, srv692413303=89, srv1043872124=3, srv1868731164=51, srv1738780175=39, srv2098929128=66, srv619746174=86, srv28291732=73, srv279782949=71, srv1770066928=43, srv243315215=67, srv1088629657=6, srv1202303595=13, srv1066502924=4, srv175260503=42, srv197677319=57, srv1202627306=14, srv1412359723=21, srv549211949=80, srv129668858=18, srv1704076044=37, srv1920122272=52, srv1505990858=27, srv1943916076=54, srv1275747967=17, srv1444374977=23, srv589633638=82, srv1992954394=59, srv1830503925=48, srv1851110860=50, srv1118780806=7, srv1784956813=44, srv1173086086=10, srv205462963=63, srv179386267=46, srv106847771=5, srv1425962419=22, srv537017713=79, srv675907970=88, srv1923500409=53, srv773201815=91, srv2063301711=64, srv275640169=70, srv1234480838=15, srv2039808084=62, srv37926455=75, srv1747873967=41, srv1042193282=2, srv900389265=97, srv971062617=98, srv592965112=83, srv1200246990=12, srv426588467=76, srv1187671331=11, srv1522688193=30, srv552184188=81, srv1613993269=34, srv1979523027=58, srv1468297931=24, srv1563542205=33, srv272476360=69, srv1236285584=16, srv1972584978=55, srv598151573=85, srv1016132481=0, srv889050897=96, srv491190656=77, srv791289425=92, srv1790854501=45, srv984734483=99, srv1518201249=29, srv1996861352=60, srv315987812=74, srv661509789=87, srv1635844033=35, srv824504377=94} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-13 20:55:41,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-13 20:55:41,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-13 20:55:41,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-13 20:55:41,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-13 20:55:41,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-13 20:55:41,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-13 20:55:41,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-13 20:55:41,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-13 20:55:41,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-13 20:55:41,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-13 20:55:41,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-13 20:55:41,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-13 20:55:41,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-13 20:55:41,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-13 20:55:41,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-13 20:55:41,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-13 20:55:41,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-13 20:55:41,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-13 20:55:41,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-13 20:55:41,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-13 20:55:41,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-13 20:55:41,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-13 20:55:41,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-13 20:55:41,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-13 20:55:41,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-13 20:55:41,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-13 20:55:41,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-13 20:55:41,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-13 20:55:41,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-13 20:55:41,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-13 20:55:41,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-13 20:55:41,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-13 20:55:41,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-13 20:55:41,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-13 20:55:41,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-13 20:55:41,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-13 20:55:41,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-13 20:55:41,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-13 20:55:41,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-13 20:55:41,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-13 20:55:41,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-13 20:55:41,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-13 20:55:41,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-13 20:55:41,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-13 20:55:41,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-13 20:55:41,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-13 20:55:41,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-13 20:55:41,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-13 20:55:41,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-13 20:55:41,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-13 20:55:41,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-13 20:55:41,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-13 20:55:41,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-13 20:55:41,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-13 20:55:41,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-13 20:55:41,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-13 20:55:41,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-13 20:55:41,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-13 20:55:41,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-13 20:55:41,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-13 20:55:41,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-13 20:55:41,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-13 20:55:41,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-13 20:55:41,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-13 20:55:41,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-13 20:55:41,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-13 20:55:41,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-13 20:55:41,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-13 20:55:41,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-13 20:55:41,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-13 20:55:41,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-13 20:55:41,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-13 20:55:41,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-13 20:55:41,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-13 20:55:41,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-13 20:55:41,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-13 20:55:41,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-13 20:55:41,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-13 20:55:41,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-13 20:55:41,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-13 20:55:41,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-13 20:55:41,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-13 20:55:41,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-13 20:55:41,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-13 20:55:41,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-13 20:55:41,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-13 20:55:41,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-13 20:55:41,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-13 20:55:41,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-13 20:55:41,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-13 20:55:41,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-13 20:55:41,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-13 20:55:41,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-13 20:55:41,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-13 20:55:41,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-13 20:55:41,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-13 20:55:41,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-13 20:55:41,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-13 20:55:41,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-13 20:55:41,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-13 20:55:41,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-13 20:55:41,505 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022513799789134994); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03386776954115048, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022513799789134994); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010166666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.042166666666666665, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-13 20:55:41,505 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-13 20:55:41,505 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.013209864337271402, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022513799789134994); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03386776954115048, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022513799789134994); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010166666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.042166666666666665, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-13 20:55:41,770 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 367 ms to try 5000 different iterations.  Found a solution that moves 949 regions; Going from a computed imbalance of 0.013209864337271402 to a new imbalance of 7.008282181348803E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006997051024024128); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008395391820274807); MoveCostFunction : (multiplier=7.0, imbalance=0.10544444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006997051024024128); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0035833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-13 20:55:41,774 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1920122272:81 , srv1972584978:82 , srv984734483:82 , srv1202627306:83 , srv1505990858:83 , srv1743987270:83 , srv279782949:83 , srv595147186:83 , srv619746174:83 , srv757152295:83 , srv971062617:83 , srv1143398149:84 , srv1236285584:84 , srv1275747967:84 , srv129668858:84 , srv1468297931:84 , srv1499646716:84 , srv1728799952:84 , srv1784956813:84 , srv1830503925:84 , srv1851110860:84 , srv2025563547:84 , srv243315215:84 , srv25383225:84 , srv272476360:84 , srv28291732:84 , srv549211949:84 , srv661509789:84 , srv79782957:84 , srv900389265:84 , srv1016132481:85 , srv106847771:85 , srv1131420607:85 , srv1187671331:85 , srv1200246990:85 , srv1202303595:85 , srv1425962419:85 , srv1511434724:85 , srv1518201249:85 , srv1522688193:85 , srv1549720844:85 , srv1747873967:85 , srv179386267:85 , srv1868731164:85 , srv1923500409:85 , srv1979523027:85 , srv1992954394:85 , srv205462963:85 , srv37926455:85 , srv426588467:85 , srv491190656:85 , srv552184188:85 , srv791289425:85 , srv860877796:85 , srv1033539982:86 , srv1042193282:86 , srv1043872124:86 , srv1088629657:86 , srv1118780806:86 , srv130901814:86 , srv1330928572:86 , srv1412359723:86 , srv1613993269:86 , srv1738780175:86 , srv175260503:86 , srv1770066928:86 , srv1790854501:86 , srv1996861352:86 , srv275640169:86 , srv28072949:86 , srv315987812:86 , srv537017713:86 , srv592965112:86 , srv692413303:86 , srv773201815:86 , srv889050897:86 , srv1066502924:87 , srv1173086086:87 , srv1444374977:87 , srv1563542205:87 , srv1802673845:87 , srv1943916076:87 , srv197629290:87 , srv2039808084:87 , srv2067919109:87 , srv589633638:87 , srv598151573:87 , srv1234480838:88 , srv1541467116:88 , srv1692584678:88 , srv1704076044:88 , srv184899678:88 , srv2063301711:88 , srv2098929128:88 , srv824504377:88 , srv1471718915:89 , srv1635844033:89 , srv197677319:89 , srv675907970:89 , srv517780356:546 }
2023-04-13 20:55:41,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-13 20:55:41,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1471718915=25, srv860877796=95, srv1499646716=26, srv28072949=72, srv25383225=68, srv1511434724=28, srv1802673845=47, srv1743987270=40, srv1330928572=20, srv1541467116=31, srv1692584678=36, srv757152295=90, srv1033539982=1, srv2067919109=65, srv184899678=49, srv595147186=84, srv79782957=93, srv130901814=19, srv517780356=78, srv1131420607=8, srv2025563547=61, srv1549720844=32, srv1143398149=9, srv1728799952=38, srv197629290=56, srv692413303=89, srv1043872124=3, srv1868731164=51, srv1738780175=39, srv2098929128=66, srv619746174=86, srv28291732=73, srv279782949=71, srv1770066928=43, srv243315215=67, srv1088629657=6, srv1202303595=13, srv1066502924=4, srv175260503=42, srv197677319=57, srv1202627306=14, srv1412359723=21, srv549211949=80, srv129668858=18, srv1704076044=37, srv1920122272=52, srv1505990858=27, srv1943916076=54, srv1275747967=17, srv1444374977=23, srv589633638=82, srv1992954394=59, srv1830503925=48, srv1851110860=50, srv1118780806=7, srv1784956813=44, srv1173086086=10, srv205462963=63, srv179386267=46, srv106847771=5, srv1425962419=22, srv537017713=79, srv675907970=88, srv1923500409=53, srv773201815=91, srv2063301711=64, srv275640169=70, srv1234480838=15, srv2039808084=62, srv37926455=75, srv1747873967=41, srv1042193282=2, srv900389265=97, srv971062617=98, srv592965112=83, srv1200246990=12, srv426588467=76, srv1187671331=11, srv1522688193=30, srv552184188=81, srv1613993269=34, srv1979523027=58, srv1468297931=24, srv1563542205=33, srv272476360=69, srv1236285584=16, srv1972584978=55, srv598151573=85, srv1016132481=0, srv889050897=96, srv491190656=77, srv791289425=92, srv1790854501=45, srv984734483=99, srv1518201249=29, srv1996861352=60, srv315987812=74, srv661509789=87, srv1635844033=35, srv824504377=94} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-13 20:55:41,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-13 20:55:41,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-13 20:55:41,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-13 20:55:41,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-13 20:55:41,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-13 20:55:41,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-13 20:55:41,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-13 20:55:41,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-13 20:55:41,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-13 20:55:41,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-13 20:55:41,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-13 20:55:41,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-13 20:55:41,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-13 20:55:41,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-13 20:55:41,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-13 20:55:41,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-13 20:55:41,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-13 20:55:41,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-13 20:55:41,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-13 20:55:41,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-13 20:55:41,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-13 20:55:41,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-13 20:55:41,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-13 20:55:41,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-13 20:55:41,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-13 20:55:41,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-13 20:55:41,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-13 20:55:41,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-13 20:55:41,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-13 20:55:41,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-13 20:55:41,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-13 20:55:41,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-13 20:55:41,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-13 20:55:41,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-13 20:55:41,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-13 20:55:41,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-13 20:55:41,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-13 20:55:41,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-13 20:55:41,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-13 20:55:41,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-13 20:55:41,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-13 20:55:41,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-13 20:55:41,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-13 20:55:41,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-13 20:55:41,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-13 20:55:41,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-13 20:55:41,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-13 20:55:41,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-13 20:55:41,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-13 20:55:41,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-13 20:55:41,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-13 20:55:41,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-13 20:55:41,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-13 20:55:41,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-13 20:55:41,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-13 20:55:41,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-13 20:55:41,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-13 20:55:41,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-13 20:55:41,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-13 20:55:41,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-13 20:55:41,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-13 20:55:41,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-13 20:55:41,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-13 20:55:41,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-13 20:55:41,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-13 20:55:41,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-13 20:55:41,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-13 20:55:41,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-13 20:55:41,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-13 20:55:41,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-13 20:55:41,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-13 20:55:41,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-13 20:55:41,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-13 20:55:41,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-13 20:55:41,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-13 20:55:41,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-13 20:55:41,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-13 20:55:41,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-13 20:55:41,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-13 20:55:41,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-13 20:55:41,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-13 20:55:41,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-13 20:55:41,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-13 20:55:41,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-13 20:55:41,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-13 20:55:41,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-13 20:55:41,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-13 20:55:41,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-13 20:55:41,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-13 20:55:41,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-13 20:55:41,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-13 20:55:41,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-13 20:55:41,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-13 20:55:41,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-13 20:55:41,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-13 20:55:41,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-13 20:55:41,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-13 20:55:41,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-13 20:55:41,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-13 20:55:41,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-13 20:55:41,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-13 20:55:41,937 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006997051024024128); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008395391820274807); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006997051024024128); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0035833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-13 20:55:41,937 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-13 20:55:41,937 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=6.94183178107592E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006997051024024128); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008395391820274807); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006997051024024128); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0035833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-13 20:55:42,182 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 346 ms to try 5000 different iterations.  Found a solution that moves 231 regions; Going from a computed imbalance of 6.94183178107592E-4 to a new imbalance of 3.55719052724192E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0020651563939657444); MoveCostFunction : (multiplier=7.0, imbalance=0.025666666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-13 20:55:42,183 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1042193282:83 , srv1066502924:83 , srv1613993269:83 , srv1033539982:84 , srv106847771:84 , srv1131420607:84 , srv1173086086:84 , srv1200246990:84 , srv1202627306:84 , srv1234480838:84 , srv1330928572:84 , srv1412359723:84 , srv1468297931:84 , srv1471718915:84 , srv1563542205:84 , srv1635844033:84 , srv1738780175:84 , srv1784956813:84 , srv1802673845:84 , srv184899678:84 , srv1851110860:84 , srv1868731164:84 , srv1943916076:84 , srv1992954394:84 , srv2025563547:84 , srv2039808084:84 , srv2067919109:84 , srv2098929128:84 , srv243315215:84 , srv275640169:84 , srv279782949:84 , srv28072949:84 , srv28291732:84 , srv37926455:84 , srv552184188:84 , srv675907970:84 , srv791289425:84 , srv824504377:84 , srv889050897:84 , srv971062617:84 , srv1016132481:85 , srv1043872124:85 , srv1088629657:85 , srv1118780806:85 , srv1143398149:85 , srv1187671331:85 , srv1202303595:85 , srv1236285584:85 , srv129668858:85 , srv130901814:85 , srv1425962419:85 , srv1444374977:85 , srv1499646716:85 , srv1505990858:85 , srv1511434724:85 , srv1518201249:85 , srv1541467116:85 , srv1549720844:85 , srv1692584678:85 , srv1704076044:85 , srv1728799952:85 , srv1743987270:85 , srv1747873967:85 , srv175260503:85 , srv1770066928:85 , srv1790854501:85 , srv179386267:85 , srv1830503925:85 , srv1923500409:85 , srv1972584978:85 , srv197629290:85 , srv197677319:85 , srv1979523027:85 , srv1996861352:85 , srv205462963:85 , srv2063301711:85 , srv272476360:85 , srv315987812:85 , srv426588467:85 , srv491190656:85 , srv537017713:85 , srv589633638:85 , srv592965112:85 , srv595147186:85 , srv598151573:85 , srv619746174:85 , srv661509789:85 , srv692413303:85 , srv757152295:85 , srv773201815:85 , srv79782957:85 , srv860877796:85 , srv984734483:85 , srv1275747967:86 , srv1522688193:86 , srv1920122272:86 , srv25383225:86 , srv549211949:86 , srv900389265:86 , srv517780356:622 }
2023-04-13 20:55:42,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-13 20:55:42,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1471718915=25, srv860877796=95, srv1499646716=26, srv28072949=72, srv25383225=68, srv1511434724=28, srv1802673845=47, srv1743987270=40, srv1330928572=20, srv1541467116=31, srv1692584678=36, srv757152295=90, srv1033539982=1, srv2067919109=65, srv184899678=49, srv595147186=84, srv79782957=93, srv130901814=19, srv517780356=78, srv1131420607=8, srv2025563547=61, srv1549720844=32, srv1143398149=9, srv1728799952=38, srv197629290=56, srv692413303=89, srv1043872124=3, srv1868731164=51, srv1738780175=39, srv2098929128=66, srv619746174=86, srv28291732=73, srv279782949=71, srv1770066928=43, srv243315215=67, srv1088629657=6, srv1202303595=13, srv1066502924=4, srv175260503=42, srv197677319=57, srv1202627306=14, srv1412359723=21, srv549211949=80, srv129668858=18, srv1704076044=37, srv1920122272=52, srv1505990858=27, srv1943916076=54, srv1275747967=17, srv1444374977=23, srv589633638=82, srv1992954394=59, srv1830503925=48, srv1851110860=50, srv1118780806=7, srv1784956813=44, srv1173086086=10, srv205462963=63, srv179386267=46, srv106847771=5, srv1425962419=22, srv537017713=79, srv675907970=88, srv1923500409=53, srv773201815=91, srv2063301711=64, srv275640169=70, srv1234480838=15, srv2039808084=62, srv37926455=75, srv1747873967=41, srv1042193282=2, srv900389265=97, srv971062617=98, srv592965112=83, srv1200246990=12, srv426588467=76, srv1187671331=11, srv1522688193=30, srv552184188=81, srv1613993269=34, srv1979523027=58, srv1468297931=24, srv1563542205=33, srv272476360=69, srv1236285584=16, srv1972584978=55, srv598151573=85, srv1016132481=0, srv889050897=96, srv491190656=77, srv791289425=92, srv1790854501=45, srv984734483=99, srv1518201249=29, srv1996861352=60, srv315987812=74, srv661509789=87, srv1635844033=35, srv824504377=94} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-13 20:55:42,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-13 20:55:42,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-13 20:55:42,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-13 20:55:42,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-13 20:55:42,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-13 20:55:42,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-13 20:55:42,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-13 20:55:42,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-13 20:55:42,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-13 20:55:42,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-13 20:55:42,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-13 20:55:42,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-13 20:55:42,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-13 20:55:42,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-13 20:55:42,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-13 20:55:42,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-13 20:55:42,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-13 20:55:42,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-13 20:55:42,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-13 20:55:42,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-13 20:55:42,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-13 20:55:42,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-13 20:55:42,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-13 20:55:42,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-13 20:55:42,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-13 20:55:42,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-13 20:55:42,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-13 20:55:42,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-13 20:55:42,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-13 20:55:42,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-13 20:55:42,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-13 20:55:42,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-13 20:55:42,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-13 20:55:42,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-13 20:55:42,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-13 20:55:42,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-13 20:55:42,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-13 20:55:42,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-13 20:55:42,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-13 20:55:42,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-13 20:55:42,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-13 20:55:42,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-13 20:55:42,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-13 20:55:42,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-13 20:55:42,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-13 20:55:42,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-13 20:55:42,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-13 20:55:42,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-13 20:55:42,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-13 20:55:42,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-13 20:55:42,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-13 20:55:42,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-13 20:55:42,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-13 20:55:42,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-13 20:55:42,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-13 20:55:42,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-13 20:55:42,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-13 20:55:42,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-13 20:55:42,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-13 20:55:42,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-13 20:55:42,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-13 20:55:42,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-13 20:55:42,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-13 20:55:42,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-13 20:55:42,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-13 20:55:42,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-13 20:55:42,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-13 20:55:42,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-13 20:55:42,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-13 20:55:42,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-13 20:55:42,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-13 20:55:42,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-13 20:55:42,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-13 20:55:42,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-13 20:55:42,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-13 20:55:42,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-13 20:55:42,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-13 20:55:42,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-13 20:55:42,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-13 20:55:42,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-13 20:55:42,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-13 20:55:42,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-13 20:55:42,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-13 20:55:42,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-13 20:55:42,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-13 20:55:42,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-13 20:55:42,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-13 20:55:42,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-13 20:55:42,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-13 20:55:42,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-13 20:55:42,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-13 20:55:42,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-13 20:55:42,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-13 20:55:42,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-13 20:55:42,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-13 20:55:42,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-13 20:55:42,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-13 20:55:42,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-13 20:55:42,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-13 20:55:42,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-13 20:55:42,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-13 20:55:42,306 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0020651563939657444); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-13 20:55:42,306 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-13 20:55:42,307 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=3.395440870097176E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0020651563939657444); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-13 20:55:42,520 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 309 ms to try 5000 different iterations.  Found a solution that moves 28 regions; Going from a computed imbalance of 3.395440870097176E-5 to a new imbalance of 8.175750307519066E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017727200782610348); MoveCostFunction : (multiplier=7.0, imbalance=0.003111111111111111); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-13 20:55:42,520 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1275747967:83 , srv1412359723:83 , srv1770066928:83 , srv197677319:83 , srv1996861352:83 , srv37926455:83 , srv549211949:83 , srv675907970:83 , srv1016132481:84 , srv1033539982:84 , srv1043872124:84 , srv1066502924:84 , srv106847771:84 , srv1088629657:84 , srv1118780806:84 , srv1131420607:84 , srv1143398149:84 , srv1173086086:84 , srv1187671331:84 , srv1200246990:84 , srv1202303595:84 , srv1202627306:84 , srv1234480838:84 , srv1236285584:84 , srv129668858:84 , srv130901814:84 , srv1330928572:84 , srv1425962419:84 , srv1444374977:84 , srv1468297931:84 , srv1471718915:84 , srv1499646716:84 , srv1505990858:84 , srv1511434724:84 , srv1518201249:84 , srv1522688193:84 , srv1541467116:84 , srv1549720844:84 , srv1563542205:84 , srv1613993269:84 , srv1635844033:84 , srv1692584678:84 , srv1704076044:84 , srv1728799952:84 , srv1738780175:84 , srv1743987270:84 , srv1747873967:84 , srv175260503:84 , srv1784956813:84 , srv1790854501:84 , srv179386267:84 , srv1802673845:84 , srv1830503925:84 , srv184899678:84 , srv1851110860:84 , srv1920122272:84 , srv1923500409:84 , srv1943916076:84 , srv1972584978:84 , srv197629290:84 , srv1979523027:84 , srv1992954394:84 , srv2025563547:84 , srv2039808084:84 , srv205462963:84 , srv2063301711:84 , srv2067919109:84 , srv25383225:84 , srv272476360:84 , srv279782949:84 , srv28072949:84 , srv28291732:84 , srv315987812:84 , srv426588467:84 , srv537017713:84 , srv552184188:84 , srv589633638:84 , srv592965112:84 , srv595147186:84 , srv598151573:84 , srv619746174:84 , srv661509789:84 , srv692413303:84 , srv757152295:84 , srv773201815:84 , srv791289425:84 , srv79782957:84 , srv860877796:84 , srv889050897:84 , srv900389265:84 , srv984734483:84 , srv1042193282:85 , srv1868731164:85 , srv2098929128:85 , srv243315215:85 , srv275640169:85 , srv491190656:85 , srv824504377:85 , srv971062617:85 , srv517780356:684 }
2023-04-13 20:55:42,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-13 20:55:42,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1471718915=25, srv860877796=95, srv1499646716=26, srv28072949=72, srv25383225=68, srv1511434724=28, srv1802673845=47, srv1743987270=40, srv1330928572=20, srv1541467116=31, srv1692584678=36, srv757152295=90, srv1033539982=1, srv2067919109=65, srv184899678=49, srv595147186=84, srv79782957=93, srv130901814=19, srv517780356=78, srv1131420607=8, srv2025563547=61, srv1549720844=32, srv1143398149=9, srv1728799952=38, srv197629290=56, srv692413303=89, srv1043872124=3, srv1868731164=51, srv1738780175=39, srv2098929128=66, srv619746174=86, srv28291732=73, srv279782949=71, srv1770066928=43, srv243315215=67, srv1088629657=6, srv1202303595=13, srv1066502924=4, srv175260503=42, srv197677319=57, srv1202627306=14, srv1412359723=21, srv549211949=80, srv129668858=18, srv1704076044=37, srv1920122272=52, srv1505990858=27, srv1943916076=54, srv1275747967=17, srv1444374977=23, srv589633638=82, srv1992954394=59, srv1830503925=48, srv1851110860=50, srv1118780806=7, srv1784956813=44, srv1173086086=10, srv205462963=63, srv179386267=46, srv106847771=5, srv1425962419=22, srv537017713=79, srv675907970=88, srv1923500409=53, srv773201815=91, srv2063301711=64, srv275640169=70, srv1234480838=15, srv2039808084=62, srv37926455=75, srv1747873967=41, srv1042193282=2, srv900389265=97, srv971062617=98, srv592965112=83, srv1200246990=12, srv426588467=76, srv1187671331=11, srv1522688193=30, srv552184188=81, srv1613993269=34, srv1979523027=58, srv1468297931=24, srv1563542205=33, srv272476360=69, srv1236285584=16, srv1972584978=55, srv598151573=85, srv1016132481=0, srv889050897=96, srv491190656=77, srv791289425=92, srv1790854501=45, srv984734483=99, srv1518201249=29, srv1996861352=60, srv315987812=74, srv661509789=87, srv1635844033=35, srv824504377=94} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-13 20:55:42,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-13 20:55:42,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-13 20:55:42,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-13 20:55:42,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-13 20:55:42,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-13 20:55:42,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-13 20:55:42,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-13 20:55:42,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-13 20:55:42,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-13 20:55:42,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-13 20:55:42,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-13 20:55:42,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-13 20:55:42,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-13 20:55:42,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-13 20:55:42,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-13 20:55:42,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-13 20:55:42,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-13 20:55:42,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-13 20:55:42,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-13 20:55:42,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-13 20:55:42,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-13 20:55:42,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-13 20:55:42,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-13 20:55:42,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-13 20:55:42,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-13 20:55:42,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-13 20:55:42,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-13 20:55:42,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-13 20:55:42,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-13 20:55:42,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-13 20:55:42,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-13 20:55:42,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-13 20:55:42,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-13 20:55:42,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-13 20:55:42,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-13 20:55:42,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-13 20:55:42,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-13 20:55:42,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-13 20:55:42,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-13 20:55:42,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-13 20:55:42,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-13 20:55:42,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-13 20:55:42,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-13 20:55:42,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-13 20:55:42,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-13 20:55:42,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-13 20:55:42,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-13 20:55:42,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-13 20:55:42,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-13 20:55:42,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-13 20:55:42,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-13 20:55:42,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-13 20:55:42,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-13 20:55:42,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-13 20:55:42,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-13 20:55:42,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-13 20:55:42,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-13 20:55:42,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-13 20:55:42,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-13 20:55:42,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-13 20:55:42,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-13 20:55:42,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-13 20:55:42,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-13 20:55:42,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-13 20:55:42,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-13 20:55:42,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-13 20:55:42,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-13 20:55:42,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-13 20:55:42,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-13 20:55:42,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-13 20:55:42,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-13 20:55:42,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-13 20:55:42,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-13 20:55:42,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-13 20:55:42,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-13 20:55:42,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-13 20:55:42,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-13 20:55:42,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-13 20:55:42,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-13 20:55:42,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-13 20:55:42,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-13 20:55:42,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-13 20:55:42,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-13 20:55:42,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-13 20:55:42,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-13 20:55:42,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-13 20:55:42,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-13 20:55:42,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-13 20:55:42,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-13 20:55:42,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-13 20:55:42,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-13 20:55:42,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-13 20:55:42,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-13 20:55:42,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-13 20:55:42,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-13 20:55:42,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-13 20:55:42,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-13 20:55:42,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-13 20:55:42,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-13 20:55:42,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-13 20:55:42,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-13 20:55:42,642 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=7.979690117040589E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017727200782610348); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-13 20:55:42,642 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1275747967:83 , srv1412359723:83 , srv1770066928:83 , srv197677319:83 , srv1996861352:83 , srv37926455:83 , srv549211949:83 , srv675907970:83 , srv1016132481:84 , srv1033539982:84 , srv1043872124:84 , srv1066502924:84 , srv106847771:84 , srv1088629657:84 , srv1118780806:84 , srv1131420607:84 , srv1143398149:84 , srv1173086086:84 , srv1187671331:84 , srv1200246990:84 , srv1202303595:84 , srv1202627306:84 , srv1234480838:84 , srv1236285584:84 , srv129668858:84 , srv130901814:84 , srv1330928572:84 , srv1425962419:84 , srv1444374977:84 , srv1468297931:84 , srv1471718915:84 , srv1499646716:84 , srv1505990858:84 , srv1511434724:84 , srv1518201249:84 , srv1522688193:84 , srv1541467116:84 , srv1549720844:84 , srv1563542205:84 , srv1613993269:84 , srv1635844033:84 , srv1692584678:84 , srv1704076044:84 , srv1728799952:84 , srv1738780175:84 , srv1743987270:84 , srv1747873967:84 , srv175260503:84 , srv1784956813:84 , srv1790854501:84 , srv179386267:84 , srv1802673845:84 , srv1830503925:84 , srv184899678:84 , srv1851110860:84 , srv1920122272:84 , srv1923500409:84 , srv1943916076:84 , srv1972584978:84 , srv197629290:84 , srv1979523027:84 , srv1992954394:84 , srv2025563547:84 , srv2039808084:84 , srv205462963:84 , srv2063301711:84 , srv2067919109:84 , srv25383225:84 , srv272476360:84 , srv279782949:84 , srv28072949:84 , srv28291732:84 , srv315987812:84 , srv426588467:84 , srv537017713:84 , srv552184188:84 , srv589633638:84 , srv592965112:84 , srv595147186:84 , srv598151573:84 , srv619746174:84 , srv661509789:84 , srv692413303:84 , srv757152295:84 , srv773201815:84 , srv791289425:84 , srv79782957:84 , srv860877796:84 , srv889050897:84 , srv900389265:84 , srv984734483:84 , srv1042193282:85 , srv1868731164:85 , srv2098929128:85 , srv243315215:85 , srv275640169:85 , srv491190656:85 , srv824504377:85 , srv971062617:85 , srv517780356:684 }
2023-04-13 20:55:42,668 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=393 (was 393), ProcessCount=173 (was 173), AvailableMemoryMB=837 (was 1329)
2023-04-13 20:55:42,675 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=393, ProcessCount=173, AvailableMemoryMB=836
2023-04-13 20:55:42,676 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-13 20:55:42,677 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-04-13 20:55:42,689 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1002905599:84 , srv1018653801:84 , srv1026866419:84 , srv1045774853:84 , srv1056678244:84 , srv1142415056:84 , srv1171432728:84 , srv1196807732:84 , srv129430509:84 , srv1300171502:84 , srv1493621510:84 , srv1519557645:84 , srv1557423854:84 , srv1591518494:84 , srv1600521992:84 , srv186636965:84 , srv1957249324:84 , srv2015958528:84 , srv2030442633:84 , srv2037500853:84 , srv226255891:84 , srv250333490:84 , srv333532840:84 , srv374685880:84 , srv564445461:84 , srv568379815:84 , srv601599094:84 , srv659828350:84 , srv961190500:84 , srv2115948894:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-04-13 20:55:42,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-13 20:55:42,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2115948894=20, srv1018653801=1, srv1056678244=4, srv1142415056=5, srv1026866419=2, srv374685880=24, srv601599094=27, srv1957249324=16, srv1300171502=9, srv2037500853=19, srv1002905599=0, srv129430509=8, srv250333490=22, srv1196807732=7, srv226255891=21, srv2015958528=17, srv1171432728=6, srv1493621510=10, srv568379815=26, srv564445461=25, srv659828350=28, srv961190500=29, srv1591518494=13, srv333532840=23, srv1519557645=11, srv186636965=15, srv2030442633=18, srv1600521992=14, srv1557423854=12, srv1045774853=3} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-13 20:55:42,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-13 20:55:42,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-13 20:55:42,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-13 20:55:42,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-13 20:55:42,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-13 20:55:42,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-13 20:55:42,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-13 20:55:42,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-13 20:55:42,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-13 20:55:42,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-13 20:55:42,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-13 20:55:42,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-13 20:55:42,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-13 20:55:42,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-13 20:55:42,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-13 20:55:42,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-13 20:55:42,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-13 20:55:42,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-13 20:55:42,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-13 20:55:42,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-13 20:55:42,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-13 20:55:42,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-13 20:55:42,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-13 20:55:42,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-13 20:55:42,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-13 20:55:42,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-13 20:55:42,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-13 20:55:42,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-13 20:55:42,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-13 20:55:42,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-13 20:55:42,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-04-13 20:55:42,733 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1488612206775228, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-13 20:55:42,733 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912662100109868, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1488612206775228, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-04-13 20:57:42,710 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6235894 different iterations.  Found a solution that moves 1849 regions; Going from a computed imbalance of 0.9912662100109868 to a new imbalance of 4.3156582404131396E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6848148148148148, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-13 20:57:42,721 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1002905599:90 , srv1018653801:90 , srv1026866419:90 , srv1045774853:90 , srv1056678244:90 , srv1142415056:90 , srv1171432728:90 , srv1196807732:90 , srv129430509:90 , srv1300171502:90 , srv1493621510:90 , srv1519557645:90 , srv1557423854:90 , srv1591518494:90 , srv1600521992:90 , srv186636965:90 , srv1957249324:90 , srv2015958528:90 , srv2030442633:90 , srv2037500853:90 , srv2115948894:90 , srv226255891:90 , srv250333490:90 , srv333532840:90 , srv374685880:90 , srv564445461:90 , srv568379815:90 , srv601599094:90 , srv659828350:90 , srv961190500:90 }
2023-04-13 20:57:42,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-13 20:57:42,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2115948894=20, srv1018653801=1, srv1056678244=4, srv1142415056=5, srv1026866419=2, srv374685880=24, srv601599094=27, srv1957249324=16, srv1300171502=9, srv2037500853=19, srv1002905599=0, srv129430509=8, srv250333490=22, srv1196807732=7, srv226255891=21, srv2015958528=17, srv1171432728=6, srv1493621510=10, srv568379815=26, srv564445461=25, srv659828350=28, srv961190500=29, srv1591518494=13, srv333532840=23, srv1519557645=11, srv186636965=15, srv2030442633=18, srv1600521992=14, srv1557423854=12, srv1045774853=3} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-13 20:57:42,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-13 20:57:42,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-13 20:57:42,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-13 20:57:42,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-13 20:57:42,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-13 20:57:42,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-13 20:57:42,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-13 20:57:42,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-13 20:57:42,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-13 20:57:42,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-13 20:57:42,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-13 20:57:42,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-13 20:57:42,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-13 20:57:42,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-13 20:57:42,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-13 20:57:42,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-13 20:57:42,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-13 20:57:42,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-13 20:57:42,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-13 20:57:42,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-13 20:57:42,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-13 20:57:42,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-13 20:57:42,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-13 20:57:42,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-13 20:57:42,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-13 20:57:42,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-13 20:57:42,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-13 20:57:42,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-13 20:57:42,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-13 20:57:42,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-13 20:57:42,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-04-13 20:57:42,742 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-13 20:57:42,742 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1002905599:90 , srv1018653801:90 , srv1026866419:90 , srv1045774853:90 , srv1056678244:90 , srv1142415056:90 , srv1171432728:90 , srv1196807732:90 , srv129430509:90 , srv1300171502:90 , srv1493621510:90 , srv1519557645:90 , srv1557423854:90 , srv1591518494:90 , srv1600521992:90 , srv186636965:90 , srv1957249324:90 , srv2015958528:90 , srv2030442633:90 , srv2037500853:90 , srv2115948894:90 , srv226255891:90 , srv250333490:90 , srv333532840:90 , srv374685880:90 , srv564445461:90 , srv568379815:90 , srv601599094:90 , srv659828350:90 , srv961190500:90 }
2023-04-13 20:57:42,761 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=380 (was 393), ProcessCount=170 (was 173), AvailableMemoryMB=1081 (was 836) - AvailableMemoryMB LEAK? -