2023-04-15 15:55:36,306 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-04-15 15:55:36,611 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-04-15 15:55:36,722 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-04-15 15:55:36,780 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-15 15:55:36,787 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-04-15 15:55:36,799 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=391, ProcessCount=169, AvailableMemoryMB=1371
2023-04-15 15:55:36,801 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-15 15:55:36,801 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-04-15 15:55:36,932 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1016079016:84 , srv1033108984:84 , srv1039440802:84 , srv1043944974:84 , srv1085272171:84 , srv1085620909:84 , srv1088706943:84 , srv112063732:84 , srv1136909230:84 , srv1175303182:84 , srv118053172:84 , srv1183256446:84 , srv1207153750:84 , srv1256621761:84 , srv1257560962:84 , srv1275135227:84 , srv1280374605:84 , srv1361831831:84 , srv1361942830:84 , srv1362118681:84 , srv1371809921:84 , srv1374942282:84 , srv143150064:84 , srv1436505665:84 , srv1480707909:84 , srv1529688053:84 , srv1584840027:84 , srv162372360:84 , srv1628266656:84 , srv1642253916:84 , srv1645111161:84 , srv1703349166:84 , srv1703846066:84 , srv1704598564:84 , srv1757955462:84 , srv1764392235:84 , srv1779367542:84 , srv1779510742:84 , srv1783153947:84 , srv1808535198:84 , srv1828647639:84 , srv1830545507:84 , srv1871218921:84 , srv1875604893:84 , srv18904856:84 , srv1898297757:84 , srv192684248:84 , srv1933198650:84 , srv1963325450:84 , srv1969279095:84 , srv2004118991:84 , srv2007341815:84 , srv2027029561:84 , srv2053077192:84 , srv2062170807:84 , srv2062221494:84 , srv2063088150:84 , srv2067526806:84 , srv2067557527:84 , srv2099042684:84 , srv2110332074:84 , srv2116510850:84 , srv254931096:84 , srv270149965:84 , srv310710449:84 , srv329858366:84 , srv340709190:84 , srv357881778:84 , srv359011161:84 , srv3671485:84 , srv404378335:84 , srv461232968:84 , srv521845241:84 , srv524785863:84 , srv542213082:84 , srv55625261:84 , srv574471114:84 , srv588997716:84 , srv601418923:84 , srv602642797:84 , srv609793368:84 , srv627527563:84 , srv668465117:84 , srv700262474:84 , srv807034125:84 , srv818740251:84 , srv824720148:84 , srv831765149:84 , srv845732223:84 , srv860482891:84 , srv87429380:84 , srv894345442:84 , srv89623267:84 , srv909535475:84 , srv930289809:84 , srv965640914:84 , srv979887275:84 , srv981675293:84 , srv985384737:84 , srv238212113:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-04-15 15:55:36,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-15 15:55:36,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2099042684=59, srv1764392235=35, srv1361831831=17, srv1085620909=5, srv238212113=62, srv1085272171=4, srv609793368=81, srv2067526806=57, srv1830545507=41, srv2027029561=52, srv807034125=85, srv1779510742=37, srv1275135227=15, srv1374942282=21, srv55625261=76, srv930289809=95, srv574471114=77, srv89623267=93, srv831765149=88, srv1175303182=9, srv357881778=68, srv818740251=86, srv668465117=83, srv1783153947=38, srv1875604893=43, srv1933198650=47, srv1257560962=14, srv2067557527=58, srv1016079016=0, srv981675293=98, srv1033108984=1, srv1436505665=23, srv909535475=94, srv1480707909=24, srv1043944974=3, srv270149965=64, srv1088706943=6, srv1361942830=18, srv1704598564=33, srv2110332074=60, srv310710449=65, srv1628266656=28, srv1963325450=48, srv1207153750=12, srv1362118681=19, srv979887275=97, srv894345442=92, srv824720148=87, srv329858366=66, srv1183256446=11, srv627527563=82, srv254931096=63, srv340709190=67, srv2004118991=50, srv542213082=75, srv588997716=78, srv461232968=72, srv87429380=91, srv1871218921=42, srv524785863=74, srv2007341815=51, srv1529688053=25, srv2053077192=53, srv1779367542=36, srv601418923=79, srv845732223=89, srv2062170807=54, srv359011161=69, srv404378335=71, srv1256621761=13, srv2116510850=61, srv192684248=46, srv965640914=96, srv1645111161=30, srv1757955462=34, srv860482891=90, srv985384737=99, srv602642797=80, srv1828647639=40, srv1584840027=26, srv1136909230=8, srv162372360=27, srv2062221494=55, srv1642253916=29, srv118053172=10, srv521845241=73, srv2063088150=56, srv1039440802=2, srv1703846066=32, srv1371809921=20, srv1280374605=16, srv3671485=70, srv1703349166=31, srv112063732=7, srv143150064=22, srv1808535198=39, srv18904856=44, srv1969279095=49, srv700262474=84, srv1898297757=45} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-15 15:55:37,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-15 15:55:37,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-15 15:55:37,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-15 15:55:37,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-15 15:55:37,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-15 15:55:37,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-15 15:55:37,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-15 15:55:37,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-15 15:55:37,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-15 15:55:37,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-15 15:55:37,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-15 15:55:37,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-15 15:55:37,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-15 15:55:37,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-15 15:55:37,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-15 15:55:37,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-15 15:55:37,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-15 15:55:37,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-15 15:55:37,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-15 15:55:37,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-15 15:55:37,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-15 15:55:37,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-15 15:55:37,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-15 15:55:37,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-15 15:55:37,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-15 15:55:37,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-15 15:55:37,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-15 15:55:37,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-15 15:55:37,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-15 15:55:37,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-15 15:55:37,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-15 15:55:37,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-15 15:55:37,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-15 15:55:37,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-15 15:55:37,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-15 15:55:37,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-15 15:55:37,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-15 15:55:37,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-15 15:55:37,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-15 15:55:37,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-15 15:55:37,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-15 15:55:37,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-15 15:55:37,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-15 15:55:37,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-15 15:55:37,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-15 15:55:37,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-15 15:55:37,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-15 15:55:37,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-15 15:55:37,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-15 15:55:37,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-15 15:55:37,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-15 15:55:37,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-15 15:55:37,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-15 15:55:37,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-15 15:55:37,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-15 15:55:37,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-15 15:55:37,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-15 15:55:37,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-15 15:55:37,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-15 15:55:37,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-15 15:55:37,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-15 15:55:37,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-15 15:55:37,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-15 15:55:37,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-15 15:55:37,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-15 15:55:37,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-15 15:55:37,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-15 15:55:37,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-15 15:55:37,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-15 15:55:37,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-15 15:55:37,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-15 15:55:37,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-15 15:55:37,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-15 15:55:37,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-15 15:55:37,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-15 15:55:37,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-15 15:55:37,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-15 15:55:37,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-15 15:55:37,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-15 15:55:37,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-15 15:55:37,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-15 15:55:37,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-15 15:55:37,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-15 15:55:37,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-15 15:55:37,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-15 15:55:37,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-15 15:55:37,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-15 15:55:37,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-15 15:55:37,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-15 15:55:37,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-15 15:55:37,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-15 15:55:37,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-15 15:55:37,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-15 15:55:37,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-15 15:55:37,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-15 15:55:37,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-15 15:55:37,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-15 15:55:37,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-15 15:55:37,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-15 15:55:37,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-15 15:55:37,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-15 15:55:37,192 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-15 15:55:37,192 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-15 15:55:37,193 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-15 15:55:37,456 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 398 ms to try 5000 different iterations.  Found a solution that moves 4690 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18083244561964149. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040374849163102265, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05215448002575093, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5211111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040374849163102265, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17658333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23766666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-15 15:55:37,465 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv2067557527:81 , srv1280374605:82 , srv1704598564:82 , srv1808535198:82 , srv1963325450:82 , srv1033108984:83 , srv1043944974:83 , srv1136909230:83 , srv1779510742:83 , srv542213082:83 , srv818740251:83 , srv1085620909:84 , srv1207153750:84 , srv1361831831:84 , srv1374942282:84 , srv143150064:84 , srv2007341815:84 , srv310710449:84 , srv588997716:84 , srv979887275:84 , srv1016079016:85 , srv112063732:85 , srv1183256446:85 , srv1361942830:85 , srv1436505665:85 , srv18904856:85 , srv1898297757:85 , srv192684248:85 , srv1933198650:85 , srv461232968:85 , srv521845241:85 , srv627527563:85 , srv668465117:85 , srv700262474:85 , srv807034125:85 , srv845732223:85 , srv894345442:85 , srv1085272171:86 , srv1275135227:86 , srv1371809921:86 , srv1480707909:86 , srv1529688053:86 , srv1645111161:86 , srv1703846066:86 , srv1828647639:86 , srv1871218921:86 , srv2027029561:86 , srv2067526806:86 , srv254931096:86 , srv404378335:86 , srv87429380:86 , srv965640914:86 , srv1039440802:87 , srv118053172:87 , srv1362118681:87 , srv162372360:87 , srv1642253916:87 , srv1830545507:87 , srv2053077192:87 , srv2062221494:87 , srv2063088150:87 , srv2099042684:87 , srv340709190:87 , srv359011161:87 , srv524785863:87 , srv824720148:87 , srv831765149:87 , srv89623267:87 , srv1088706943:88 , srv1256621761:88 , srv1257560962:88 , srv1628266656:88 , srv1779367542:88 , srv1783153947:88 , srv1969279095:88 , srv329858366:88 , srv357881778:88 , srv55625261:88 , srv601418923:88 , srv602642797:88 , srv1175303182:89 , srv1584840027:89 , srv1703349166:89 , srv1757955462:89 , srv1764392235:89 , srv2004118991:89 , srv2116510850:89 , srv270149965:89 , srv3671485:89 , srv609793368:89 , srv860482891:89 , srv909535475:89 , srv985384737:89 , srv930289809:90 , srv1875604893:91 , srv981675293:91 , srv2062170807:92 , srv2110332074:92 , srv574471114:92 , srv238212113:449 }
2023-04-15 15:55:37,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-15 15:55:37,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2099042684=59, srv1764392235=35, srv1361831831=17, srv1085620909=5, srv238212113=62, srv1085272171=4, srv609793368=81, srv2067526806=57, srv1830545507=41, srv2027029561=52, srv807034125=85, srv1779510742=37, srv1275135227=15, srv1374942282=21, srv55625261=76, srv930289809=95, srv574471114=77, srv89623267=93, srv831765149=88, srv1175303182=9, srv357881778=68, srv818740251=86, srv668465117=83, srv1783153947=38, srv1875604893=43, srv1933198650=47, srv1257560962=14, srv2067557527=58, srv1016079016=0, srv981675293=98, srv1033108984=1, srv1436505665=23, srv909535475=94, srv1480707909=24, srv1043944974=3, srv270149965=64, srv1088706943=6, srv1361942830=18, srv1704598564=33, srv2110332074=60, srv310710449=65, srv1628266656=28, srv1963325450=48, srv1207153750=12, srv1362118681=19, srv979887275=97, srv894345442=92, srv824720148=87, srv329858366=66, srv1183256446=11, srv627527563=82, srv254931096=63, srv340709190=67, srv2004118991=50, srv542213082=75, srv588997716=78, srv461232968=72, srv87429380=91, srv1871218921=42, srv524785863=74, srv2007341815=51, srv1529688053=25, srv2053077192=53, srv1779367542=36, srv601418923=79, srv845732223=89, srv2062170807=54, srv359011161=69, srv404378335=71, srv1256621761=13, srv2116510850=61, srv192684248=46, srv965640914=96, srv1645111161=30, srv1757955462=34, srv860482891=90, srv985384737=99, srv602642797=80, srv1828647639=40, srv1584840027=26, srv1136909230=8, srv162372360=27, srv2062221494=55, srv1642253916=29, srv118053172=10, srv521845241=73, srv2063088150=56, srv1039440802=2, srv1703846066=32, srv1371809921=20, srv1280374605=16, srv3671485=70, srv1703349166=31, srv112063732=7, srv143150064=22, srv1808535198=39, srv18904856=44, srv1969279095=49, srv700262474=84, srv1898297757=45} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-15 15:55:37,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-15 15:55:37,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-15 15:55:37,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-15 15:55:37,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-15 15:55:37,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-15 15:55:37,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-15 15:55:37,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-15 15:55:37,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-15 15:55:37,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-15 15:55:37,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-15 15:55:37,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-15 15:55:37,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-15 15:55:37,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-15 15:55:37,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-15 15:55:37,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-15 15:55:37,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-15 15:55:37,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-15 15:55:37,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-15 15:55:37,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-15 15:55:37,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-15 15:55:37,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-15 15:55:37,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-15 15:55:37,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-15 15:55:37,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-15 15:55:37,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-15 15:55:37,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-15 15:55:37,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-15 15:55:37,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-15 15:55:37,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-15 15:55:37,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-15 15:55:37,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-15 15:55:37,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-15 15:55:37,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-15 15:55:37,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-15 15:55:37,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-15 15:55:37,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-15 15:55:37,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-15 15:55:37,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-15 15:55:37,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-15 15:55:37,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-15 15:55:37,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-15 15:55:37,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-15 15:55:37,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-15 15:55:37,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-15 15:55:37,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-15 15:55:37,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-15 15:55:37,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-15 15:55:37,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-15 15:55:37,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-15 15:55:37,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-15 15:55:37,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-15 15:55:37,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-15 15:55:37,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-15 15:55:37,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-15 15:55:37,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-15 15:55:37,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-15 15:55:37,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-15 15:55:37,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-15 15:55:37,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-15 15:55:37,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-15 15:55:37,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-15 15:55:37,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-15 15:55:37,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-15 15:55:37,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-15 15:55:37,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-15 15:55:37,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-15 15:55:37,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-15 15:55:37,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-15 15:55:37,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-15 15:55:37,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-15 15:55:37,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-15 15:55:37,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-15 15:55:37,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-15 15:55:37,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-15 15:55:37,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-15 15:55:37,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-15 15:55:37,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-15 15:55:37,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-15 15:55:37,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-15 15:55:37,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-15 15:55:37,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-15 15:55:37,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-15 15:55:37,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-15 15:55:37,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-15 15:55:37,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-15 15:55:37,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-15 15:55:37,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-15 15:55:37,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-15 15:55:37,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-15 15:55:37,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-15 15:55:37,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-15 15:55:37,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-15 15:55:37,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-15 15:55:37,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-15 15:55:37,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-15 15:55:37,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-15 15:55:37,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-15 15:55:37,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-15 15:55:37,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-15 15:55:37,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-15 15:55:37,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-15 15:55:37,617 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040374849163102265, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05215448002575093, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040374849163102265, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17658333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23766666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-15 15:55:37,617 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-15 15:55:37,617 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18079960553773633, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040374849163102265, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05215448002575093, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040374849163102265, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17658333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23766666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-15 15:55:37,872 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 354 ms to try 5000 different iterations.  Found a solution that moves 2653 regions; Going from a computed imbalance of 0.18079960553773633 to a new imbalance of 0.013498885787851229. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020920934418537717); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.028985851264244543, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2947777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020920934418537717); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010416666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-15 15:55:37,881 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv2062170807:79 , srv2110332074:80 , srv981675293:80 , srv574471114:81 , srv1256621761:82 , srv1875604893:82 , srv2004118991:82 , srv3671485:82 , srv55625261:82 , srv1088706943:83 , srv1703349166:83 , srv1757955462:83 , srv1764392235:83 , srv1830545507:83 , srv270149965:83 , srv357881778:83 , srv524785863:83 , srv930289809:83 , srv985384737:83 , srv1085272171:84 , srv1175303182:84 , srv118053172:84 , srv1584840027:84 , srv1779367542:84 , srv2027029561:84 , srv602642797:84 , srv824720148:84 , srv909535475:84 , srv1039440802:85 , srv1361942830:85 , srv1362118681:85 , srv1529688053:85 , srv1628266656:85 , srv1703846066:85 , srv1783153947:85 , srv1969279095:85 , srv2116510850:85 , srv329858366:85 , srv340709190:85 , srv359011161:85 , srv404378335:85 , srv601418923:85 , srv609793368:85 , srv831765149:85 , srv87429380:85 , srv894345442:85 , srv112063732:86 , srv1183256446:86 , srv143150064:86 , srv1480707909:86 , srv162372360:86 , srv1642253916:86 , srv1645111161:86 , srv1828647639:86 , srv18904856:86 , srv1933198650:86 , srv2053077192:86 , srv2062221494:86 , srv2067526806:86 , srv2099042684:86 , srv521845241:86 , srv700262474:86 , srv807034125:86 , srv860482891:86 , srv89623267:86 , srv1016079016:87 , srv1257560962:87 , srv1361831831:87 , srv1374942282:87 , srv1898297757:87 , srv192684248:87 , srv2063088150:87 , srv254931096:87 , srv310710449:87 , srv965640914:87 , srv1207153750:88 , srv1275135227:88 , srv1371809921:88 , srv1779510742:88 , srv1871218921:88 , srv818740251:88 , srv845732223:88 , srv979887275:88 , srv1033108984:89 , srv1085620909:89 , srv1280374605:89 , srv1436505665:89 , srv2007341815:89 , srv461232968:89 , srv588997716:89 , srv627527563:89 , srv668465117:89 , srv1043944974:90 , srv1808535198:90 , srv2067557527:90 , srv542213082:90 , srv1136909230:91 , srv1963325450:91 , srv1704598564:92 , srv238212113:511 }
2023-04-15 15:55:37,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-15 15:55:37,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2099042684=59, srv1764392235=35, srv1361831831=17, srv1085620909=5, srv238212113=62, srv1085272171=4, srv609793368=81, srv2067526806=57, srv1830545507=41, srv2027029561=52, srv807034125=85, srv1779510742=37, srv1275135227=15, srv1374942282=21, srv55625261=76, srv930289809=95, srv574471114=77, srv89623267=93, srv831765149=88, srv1175303182=9, srv357881778=68, srv818740251=86, srv668465117=83, srv1783153947=38, srv1875604893=43, srv1933198650=47, srv1257560962=14, srv2067557527=58, srv1016079016=0, srv981675293=98, srv1033108984=1, srv1436505665=23, srv909535475=94, srv1480707909=24, srv1043944974=3, srv270149965=64, srv1088706943=6, srv1361942830=18, srv1704598564=33, srv2110332074=60, srv310710449=65, srv1628266656=28, srv1963325450=48, srv1207153750=12, srv1362118681=19, srv979887275=97, srv894345442=92, srv824720148=87, srv329858366=66, srv1183256446=11, srv627527563=82, srv254931096=63, srv340709190=67, srv2004118991=50, srv542213082=75, srv588997716=78, srv461232968=72, srv87429380=91, srv1871218921=42, srv524785863=74, srv2007341815=51, srv1529688053=25, srv2053077192=53, srv1779367542=36, srv601418923=79, srv845732223=89, srv2062170807=54, srv359011161=69, srv404378335=71, srv1256621761=13, srv2116510850=61, srv192684248=46, srv965640914=96, srv1645111161=30, srv1757955462=34, srv860482891=90, srv985384737=99, srv602642797=80, srv1828647639=40, srv1584840027=26, srv1136909230=8, srv162372360=27, srv2062221494=55, srv1642253916=29, srv118053172=10, srv521845241=73, srv2063088150=56, srv1039440802=2, srv1703846066=32, srv1371809921=20, srv1280374605=16, srv3671485=70, srv1703349166=31, srv112063732=7, srv143150064=22, srv1808535198=39, srv18904856=44, srv1969279095=49, srv700262474=84, srv1898297757=45} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-15 15:55:37,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-15 15:55:37,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-15 15:55:37,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-15 15:55:37,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-15 15:55:37,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-15 15:55:37,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-15 15:55:37,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-15 15:55:37,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-15 15:55:37,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-15 15:55:37,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-15 15:55:37,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-15 15:55:37,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-15 15:55:37,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-15 15:55:37,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-15 15:55:37,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-15 15:55:37,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-15 15:55:37,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-15 15:55:37,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-15 15:55:37,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-15 15:55:37,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-15 15:55:37,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-15 15:55:37,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-15 15:55:37,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-15 15:55:37,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-15 15:55:37,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-15 15:55:37,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-15 15:55:37,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-15 15:55:37,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-15 15:55:37,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-15 15:55:37,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-15 15:55:37,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-15 15:55:37,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-15 15:55:37,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-15 15:55:37,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-15 15:55:37,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-15 15:55:37,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-15 15:55:37,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-15 15:55:37,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-15 15:55:37,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-15 15:55:37,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-15 15:55:37,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-15 15:55:37,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-15 15:55:37,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-15 15:55:37,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-15 15:55:37,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-15 15:55:37,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-15 15:55:37,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-15 15:55:37,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-15 15:55:37,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-15 15:55:37,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-15 15:55:37,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-15 15:55:37,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-15 15:55:37,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-15 15:55:37,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-15 15:55:37,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-15 15:55:37,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-15 15:55:37,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-15 15:55:37,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-15 15:55:37,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-15 15:55:37,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-15 15:55:37,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-15 15:55:37,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-15 15:55:37,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-15 15:55:37,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-15 15:55:37,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-15 15:55:37,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-15 15:55:37,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-15 15:55:37,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-15 15:55:37,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-15 15:55:37,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-15 15:55:37,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-15 15:55:37,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-15 15:55:37,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-15 15:55:37,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-15 15:55:37,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-15 15:55:37,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-15 15:55:37,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-15 15:55:37,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-15 15:55:37,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-15 15:55:37,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-15 15:55:37,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-15 15:55:37,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-15 15:55:37,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-15 15:55:37,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-15 15:55:37,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-15 15:55:37,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-15 15:55:37,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-15 15:55:37,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-15 15:55:37,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-15 15:55:37,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-15 15:55:37,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-15 15:55:37,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-15 15:55:37,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-15 15:55:37,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-15 15:55:37,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-15 15:55:37,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-15 15:55:37,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-15 15:55:37,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-15 15:55:37,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-15 15:55:37,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-15 15:55:37,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-15 15:55:38,038 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020920934418537717); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.028985851264244543, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020920934418537717); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010416666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-15 15:55:38,039 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-15 15:55:38,039 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.013480309084803393, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020920934418537717); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.028985851264244543, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020920934418537717); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010416666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-15 15:55:38,352 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 413 ms to try 5000 different iterations.  Found a solution that moves 992 regions; Going from a computed imbalance of 0.013480309084803393 to a new imbalance of 3.423227314142006E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005658970147920401); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0067834882385377025); MoveCostFunction : (multiplier=7.0, imbalance=0.11022222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005658970147920401); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0030833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-15 15:55:38,354 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1257560962:82 , srv860482891:82 , srv1436505665:83 , srv1871218921:83 , srv1969279095:83 , srv2062221494:83 , srv574471114:83 , srv609793368:83 , srv627527563:83 , srv1136909230:84 , srv1275135227:84 , srv1371809921:84 , srv1584840027:84 , srv162372360:84 , srv1704598564:84 , srv1783153947:84 , srv1875604893:84 , srv1963325450:84 , srv2007341815:84 , srv2053077192:84 , srv2063088150:84 , srv2116510850:84 , srv461232968:84 , srv588997716:84 , srv668465117:84 , srv930289809:84 , srv965640914:84 , srv1016079016:85 , srv1033108984:85 , srv1085620909:85 , srv1175303182:85 , srv1207153750:85 , srv1480707909:85 , srv1628266656:85 , srv1642253916:85 , srv1645111161:85 , srv1703349166:85 , srv1757955462:85 , srv1764392235:85 , srv1828647639:85 , srv1898297757:85 , srv2004118991:85 , srv2067526806:85 , srv2099042684:85 , srv2110332074:85 , srv254931096:85 , srv329858366:85 , srv542213082:85 , srv601418923:85 , srv602642797:85 , srv845732223:85 , srv89623267:85 , srv909535475:85 , srv1039440802:86 , srv1043944974:86 , srv118053172:86 , srv1361831831:86 , srv1362118681:86 , srv1374942282:86 , srv1529688053:86 , srv1703846066:86 , srv1779510742:86 , srv1808535198:86 , srv18904856:86 , srv192684248:86 , srv2062170807:86 , srv270149965:86 , srv310710449:86 , srv340709190:86 , srv357881778:86 , srv359011161:86 , srv521845241:86 , srv807034125:86 , srv824720148:86 , srv979887275:86 , srv981675293:86 , srv985384737:86 , srv1088706943:87 , srv112063732:87 , srv1183256446:87 , srv1280374605:87 , srv1361942830:87 , srv143150064:87 , srv1779367542:87 , srv1830545507:87 , srv1933198650:87 , srv2067557527:87 , srv3671485:87 , srv700262474:87 , srv818740251:87 , srv831765149:87 , srv87429380:87 , srv894345442:87 , srv1256621761:88 , srv2027029561:88 , srv404378335:88 , srv524785863:88 , srv55625261:88 , srv1085272171:89 , srv238212113:548 }
2023-04-15 15:55:38,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-15 15:55:38,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2099042684=59, srv1764392235=35, srv1361831831=17, srv1085620909=5, srv238212113=62, srv1085272171=4, srv609793368=81, srv2067526806=57, srv1830545507=41, srv2027029561=52, srv807034125=85, srv1779510742=37, srv1275135227=15, srv1374942282=21, srv55625261=76, srv930289809=95, srv574471114=77, srv89623267=93, srv831765149=88, srv1175303182=9, srv357881778=68, srv818740251=86, srv668465117=83, srv1783153947=38, srv1875604893=43, srv1933198650=47, srv1257560962=14, srv2067557527=58, srv1016079016=0, srv981675293=98, srv1033108984=1, srv1436505665=23, srv909535475=94, srv1480707909=24, srv1043944974=3, srv270149965=64, srv1088706943=6, srv1361942830=18, srv1704598564=33, srv2110332074=60, srv310710449=65, srv1628266656=28, srv1963325450=48, srv1207153750=12, srv1362118681=19, srv979887275=97, srv894345442=92, srv824720148=87, srv329858366=66, srv1183256446=11, srv627527563=82, srv254931096=63, srv340709190=67, srv2004118991=50, srv542213082=75, srv588997716=78, srv461232968=72, srv87429380=91, srv1871218921=42, srv524785863=74, srv2007341815=51, srv1529688053=25, srv2053077192=53, srv1779367542=36, srv601418923=79, srv845732223=89, srv2062170807=54, srv359011161=69, srv404378335=71, srv1256621761=13, srv2116510850=61, srv192684248=46, srv965640914=96, srv1645111161=30, srv1757955462=34, srv860482891=90, srv985384737=99, srv602642797=80, srv1828647639=40, srv1584840027=26, srv1136909230=8, srv162372360=27, srv2062221494=55, srv1642253916=29, srv118053172=10, srv521845241=73, srv2063088150=56, srv1039440802=2, srv1703846066=32, srv1371809921=20, srv1280374605=16, srv3671485=70, srv1703349166=31, srv112063732=7, srv143150064=22, srv1808535198=39, srv18904856=44, srv1969279095=49, srv700262474=84, srv1898297757=45} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-15 15:55:38,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-15 15:55:38,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-15 15:55:38,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-15 15:55:38,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-15 15:55:38,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-15 15:55:38,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-15 15:55:38,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-15 15:55:38,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-15 15:55:38,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-15 15:55:38,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-15 15:55:38,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-15 15:55:38,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-15 15:55:38,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-15 15:55:38,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-15 15:55:38,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-15 15:55:38,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-15 15:55:38,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-15 15:55:38,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-15 15:55:38,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-15 15:55:38,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-15 15:55:38,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-15 15:55:38,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-15 15:55:38,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-15 15:55:38,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-15 15:55:38,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-15 15:55:38,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-15 15:55:38,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-15 15:55:38,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-15 15:55:38,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-15 15:55:38,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-15 15:55:38,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-15 15:55:38,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-15 15:55:38,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-15 15:55:38,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-15 15:55:38,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-15 15:55:38,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-15 15:55:38,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-15 15:55:38,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-15 15:55:38,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-15 15:55:38,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-15 15:55:38,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-15 15:55:38,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-15 15:55:38,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-15 15:55:38,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-15 15:55:38,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-15 15:55:38,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-15 15:55:38,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-15 15:55:38,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-15 15:55:38,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-15 15:55:38,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-15 15:55:38,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-15 15:55:38,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-15 15:55:38,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-15 15:55:38,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-15 15:55:38,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-15 15:55:38,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-15 15:55:38,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-15 15:55:38,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-15 15:55:38,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-15 15:55:38,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-15 15:55:38,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-15 15:55:38,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-15 15:55:38,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-15 15:55:38,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-15 15:55:38,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-15 15:55:38,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-15 15:55:38,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-15 15:55:38,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-15 15:55:38,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-15 15:55:38,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-15 15:55:38,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-15 15:55:38,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-15 15:55:38,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-15 15:55:38,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-15 15:55:38,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-15 15:55:38,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-15 15:55:38,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-15 15:55:38,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-15 15:55:38,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-15 15:55:38,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-15 15:55:38,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-15 15:55:38,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-15 15:55:38,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-15 15:55:38,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-15 15:55:38,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-15 15:55:38,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-15 15:55:38,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-15 15:55:38,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-15 15:55:38,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-15 15:55:38,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-15 15:55:38,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-15 15:55:38,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-15 15:55:38,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-15 15:55:38,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-15 15:55:38,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-15 15:55:38,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-15 15:55:38,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-15 15:55:38,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-15 15:55:38,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-15 15:55:38,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-15 15:55:38,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-15 15:55:38,495 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005658970147920401); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0067834882385377025); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005658970147920401); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0030833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-15 15:55:38,495 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-15 15:55:38,495 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=3.353765989515345E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005658970147920401); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0067834882385377025); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005658970147920401); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0030833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-15 15:55:38,764 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 364 ms to try 5000 different iterations.  Found a solution that moves 234 regions; Going from a computed imbalance of 3.353765989515345E-4 to a new imbalance of 1.839176664984743E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.868392354571063E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0016412198797244363); MoveCostFunction : (multiplier=7.0, imbalance=0.026, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.868392354571063E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-15 15:55:38,765 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1043944974:83 , srv1085272171:83 , srv1779367542:83 , srv404378335:83 , srv524785863:83 , srv831765149:83 , srv1039440802:84 , srv1085620909:84 , srv1088706943:84 , srv112063732:84 , srv1136909230:84 , srv1175303182:84 , srv118053172:84 , srv1183256446:84 , srv1256621761:84 , srv1275135227:84 , srv1361942830:84 , srv1362118681:84 , srv1371809921:84 , srv1584840027:84 , srv1628266656:84 , srv1642253916:84 , srv1645111161:84 , srv1704598564:84 , srv1808535198:84 , srv192684248:84 , srv1933198650:84 , srv1963325450:84 , srv2027029561:84 , srv2063088150:84 , srv2067557527:84 , srv2099042684:84 , srv2116510850:84 , srv254931096:84 , srv270149965:84 , srv329858366:84 , srv340709190:84 , srv359011161:84 , srv3671485:84 , srv461232968:84 , srv542213082:84 , srv700262474:84 , srv818740251:84 , srv845732223:84 , srv909535475:84 , srv979887275:84 , srv985384737:84 , srv1016079016:85 , srv1033108984:85 , srv1207153750:85 , srv1257560962:85 , srv1280374605:85 , srv1361831831:85 , srv1374942282:85 , srv143150064:85 , srv1436505665:85 , srv1480707909:85 , srv1529688053:85 , srv162372360:85 , srv1703349166:85 , srv1703846066:85 , srv1757955462:85 , srv1764392235:85 , srv1779510742:85 , srv1783153947:85 , srv1828647639:85 , srv1830545507:85 , srv1871218921:85 , srv1875604893:85 , srv18904856:85 , srv1898297757:85 , srv2007341815:85 , srv2053077192:85 , srv2062170807:85 , srv2067526806:85 , srv2110332074:85 , srv310710449:85 , srv357881778:85 , srv521845241:85 , srv55625261:85 , srv588997716:85 , srv601418923:85 , srv602642797:85 , srv609793368:85 , srv627527563:85 , srv668465117:85 , srv807034125:85 , srv824720148:85 , srv860482891:85 , srv87429380:85 , srv894345442:85 , srv89623267:85 , srv930289809:85 , srv965640914:85 , srv981675293:85 , srv1969279095:86 , srv2004118991:86 , srv2062221494:86 , srv574471114:86 , srv238212113:634 }
2023-04-15 15:55:38,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-15 15:55:38,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2099042684=59, srv1764392235=35, srv1361831831=17, srv1085620909=5, srv238212113=62, srv1085272171=4, srv609793368=81, srv2067526806=57, srv1830545507=41, srv2027029561=52, srv807034125=85, srv1779510742=37, srv1275135227=15, srv1374942282=21, srv55625261=76, srv930289809=95, srv574471114=77, srv89623267=93, srv831765149=88, srv1175303182=9, srv357881778=68, srv818740251=86, srv668465117=83, srv1783153947=38, srv1875604893=43, srv1933198650=47, srv1257560962=14, srv2067557527=58, srv1016079016=0, srv981675293=98, srv1033108984=1, srv1436505665=23, srv909535475=94, srv1480707909=24, srv1043944974=3, srv270149965=64, srv1088706943=6, srv1361942830=18, srv1704598564=33, srv2110332074=60, srv310710449=65, srv1628266656=28, srv1963325450=48, srv1207153750=12, srv1362118681=19, srv979887275=97, srv894345442=92, srv824720148=87, srv329858366=66, srv1183256446=11, srv627527563=82, srv254931096=63, srv340709190=67, srv2004118991=50, srv542213082=75, srv588997716=78, srv461232968=72, srv87429380=91, srv1871218921=42, srv524785863=74, srv2007341815=51, srv1529688053=25, srv2053077192=53, srv1779367542=36, srv601418923=79, srv845732223=89, srv2062170807=54, srv359011161=69, srv404378335=71, srv1256621761=13, srv2116510850=61, srv192684248=46, srv965640914=96, srv1645111161=30, srv1757955462=34, srv860482891=90, srv985384737=99, srv602642797=80, srv1828647639=40, srv1584840027=26, srv1136909230=8, srv162372360=27, srv2062221494=55, srv1642253916=29, srv118053172=10, srv521845241=73, srv2063088150=56, srv1039440802=2, srv1703846066=32, srv1371809921=20, srv1280374605=16, srv3671485=70, srv1703349166=31, srv112063732=7, srv143150064=22, srv1808535198=39, srv18904856=44, srv1969279095=49, srv700262474=84, srv1898297757=45} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-15 15:55:38,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-15 15:55:38,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-15 15:55:38,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-15 15:55:38,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-15 15:55:38,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-15 15:55:38,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-15 15:55:38,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-15 15:55:38,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-15 15:55:38,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-15 15:55:38,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-15 15:55:38,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-15 15:55:38,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-15 15:55:38,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-15 15:55:38,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-15 15:55:38,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-15 15:55:38,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-15 15:55:38,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-15 15:55:38,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-15 15:55:38,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-15 15:55:38,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-15 15:55:38,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-15 15:55:38,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-15 15:55:38,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-15 15:55:38,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-15 15:55:38,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-15 15:55:38,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-15 15:55:38,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-15 15:55:38,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-15 15:55:38,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-15 15:55:38,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-15 15:55:38,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-15 15:55:38,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-15 15:55:38,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-15 15:55:38,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-15 15:55:38,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-15 15:55:38,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-15 15:55:38,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-15 15:55:38,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-15 15:55:38,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-15 15:55:38,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-15 15:55:38,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-15 15:55:38,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-15 15:55:38,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-15 15:55:38,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-15 15:55:38,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-15 15:55:38,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-15 15:55:38,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-15 15:55:38,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-15 15:55:38,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-15 15:55:38,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-15 15:55:38,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-15 15:55:38,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-15 15:55:38,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-15 15:55:38,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-15 15:55:38,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-15 15:55:38,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-15 15:55:38,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-15 15:55:38,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-15 15:55:38,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-15 15:55:38,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-15 15:55:38,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-15 15:55:38,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-15 15:55:38,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-15 15:55:38,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-15 15:55:38,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-15 15:55:38,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-15 15:55:38,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-15 15:55:38,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-15 15:55:38,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-15 15:55:38,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-15 15:55:38,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-15 15:55:38,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-15 15:55:38,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-15 15:55:38,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-15 15:55:38,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-15 15:55:38,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-15 15:55:38,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-15 15:55:38,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-15 15:55:38,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-15 15:55:38,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-15 15:55:38,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-15 15:55:38,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-15 15:55:38,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-15 15:55:38,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-15 15:55:38,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-15 15:55:38,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-15 15:55:38,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-15 15:55:38,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-15 15:55:38,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-15 15:55:38,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-15 15:55:38,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-15 15:55:38,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-15 15:55:38,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-15 15:55:38,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-15 15:55:38,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-15 15:55:38,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-15 15:55:38,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-15 15:55:38,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-15 15:55:38,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-15 15:55:38,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-15 15:55:38,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-15 15:55:38,893 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.675326362942016E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.868392354571063E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0016412198797244363); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.868392354571063E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-15 15:55:38,894 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1043944974:83 , srv1085272171:83 , srv1779367542:83 , srv404378335:83 , srv524785863:83 , srv831765149:83 , srv1039440802:84 , srv1085620909:84 , srv1088706943:84 , srv112063732:84 , srv1136909230:84 , srv1175303182:84 , srv118053172:84 , srv1183256446:84 , srv1256621761:84 , srv1275135227:84 , srv1361942830:84 , srv1362118681:84 , srv1371809921:84 , srv1584840027:84 , srv1628266656:84 , srv1642253916:84 , srv1645111161:84 , srv1704598564:84 , srv1808535198:84 , srv192684248:84 , srv1933198650:84 , srv1963325450:84 , srv2027029561:84 , srv2063088150:84 , srv2067557527:84 , srv2099042684:84 , srv2116510850:84 , srv254931096:84 , srv270149965:84 , srv329858366:84 , srv340709190:84 , srv359011161:84 , srv3671485:84 , srv461232968:84 , srv542213082:84 , srv700262474:84 , srv818740251:84 , srv845732223:84 , srv909535475:84 , srv979887275:84 , srv985384737:84 , srv1016079016:85 , srv1033108984:85 , srv1207153750:85 , srv1257560962:85 , srv1280374605:85 , srv1361831831:85 , srv1374942282:85 , srv143150064:85 , srv1436505665:85 , srv1480707909:85 , srv1529688053:85 , srv162372360:85 , srv1703349166:85 , srv1703846066:85 , srv1757955462:85 , srv1764392235:85 , srv1779510742:85 , srv1783153947:85 , srv1828647639:85 , srv1830545507:85 , srv1871218921:85 , srv1875604893:85 , srv18904856:85 , srv1898297757:85 , srv2007341815:85 , srv2053077192:85 , srv2062170807:85 , srv2067526806:85 , srv2110332074:85 , srv310710449:85 , srv357881778:85 , srv521845241:85 , srv55625261:85 , srv588997716:85 , srv601418923:85 , srv602642797:85 , srv609793368:85 , srv627527563:85 , srv668465117:85 , srv807034125:85 , srv824720148:85 , srv860482891:85 , srv87429380:85 , srv894345442:85 , srv89623267:85 , srv930289809:85 , srv965640914:85 , srv981675293:85 , srv1969279095:86 , srv2004118991:86 , srv2062221494:86 , srv574471114:86 , srv238212113:634 }
2023-04-15 15:55:38,920 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=384 (was 391), ProcessCount=169 (was 169), AvailableMemoryMB=886 (was 1371)
2023-04-15 15:55:38,929 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=384, ProcessCount=169, AvailableMemoryMB=886
2023-04-15 15:55:38,930 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-15 15:55:38,930 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-04-15 15:55:38,938 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1043398963:84 , srv1124397980:84 , srv1155865125:84 , srv1177656639:84 , srv1331224046:84 , srv135214300:84 , srv136138055:84 , srv1619111420:84 , srv1632622053:84 , srv1687314384:84 , srv170805063:84 , srv1790406042:84 , srv1828179758:84 , srv1896543746:84 , srv1929261082:84 , srv1931777206:84 , srv1990144017:84 , srv2013578581:84 , srv2036600168:84 , srv321476406:84 , srv474108304:84 , srv554946772:84 , srv65065163:84 , srv675586104:84 , srv678078786:84 , srv721206205:84 , srv723654571:84 , srv785840451:84 , srv811688280:84 , srv876787162:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-04-15 15:55:38,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-15 15:55:38,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv135214300=5, srv1896543746=13, srv811688280=28, srv1790406042=11, srv876787162=29, srv675586104=23, srv65065163=22, srv1929261082=14, srv2036600168=18, srv723654571=26, srv721206205=25, srv1124397980=1, srv1177656639=3, srv785840451=27, srv1043398963=0, srv1331224046=4, srv1687314384=9, srv1632622053=8, srv1619111420=7, srv1155865125=2, srv1931777206=15, srv321476406=19, srv554946772=21, srv678078786=24, srv170805063=10, srv1828179758=12, srv1990144017=16, srv2013578581=17, srv136138055=6, srv474108304=20} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-15 15:55:38,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-15 15:55:38,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-15 15:55:38,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-15 15:55:38,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-15 15:55:38,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-15 15:55:38,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-15 15:55:38,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-15 15:55:38,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-15 15:55:38,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-15 15:55:38,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-15 15:55:38,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-15 15:55:38,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-15 15:55:38,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-15 15:55:38,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-15 15:55:38,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-15 15:55:38,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-15 15:55:38,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-15 15:55:38,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-15 15:55:38,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-15 15:55:38,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-15 15:55:38,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-15 15:55:38,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-15 15:55:38,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-15 15:55:38,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-15 15:55:38,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-15 15:55:38,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-15 15:55:38,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-15 15:55:38,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-15 15:55:38,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-15 15:55:38,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-15 15:55:38,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-04-15 15:55:38,967 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1539522884824467, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-15 15:55:38,967 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912678141898282, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1539522884824467, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-04-15 15:57:38,952 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6114417 different iterations.  Found a solution that moves 1854 regions; Going from a computed imbalance of 0.9912678141898282 to a new imbalance of 4.3273284898463826E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6866666666666666, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-15 15:57:38,956 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1043398963:90 , srv1124397980:90 , srv1155865125:90 , srv1177656639:90 , srv1331224046:90 , srv135214300:90 , srv136138055:90 , srv1619111420:90 , srv1632622053:90 , srv1687314384:90 , srv170805063:90 , srv1790406042:90 , srv1828179758:90 , srv1896543746:90 , srv1929261082:90 , srv1931777206:90 , srv1990144017:90 , srv2013578581:90 , srv2036600168:90 , srv321476406:90 , srv474108304:90 , srv554946772:90 , srv65065163:90 , srv675586104:90 , srv678078786:90 , srv721206205:90 , srv723654571:90 , srv785840451:90 , srv811688280:90 , srv876787162:90 }
2023-04-15 15:57:38,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-15 15:57:38,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv135214300=5, srv1896543746=13, srv811688280=28, srv1790406042=11, srv876787162=29, srv675586104=23, srv65065163=22, srv1929261082=14, srv2036600168=18, srv723654571=26, srv721206205=25, srv1124397980=1, srv1177656639=3, srv785840451=27, srv1043398963=0, srv1331224046=4, srv1687314384=9, srv1632622053=8, srv1619111420=7, srv1155865125=2, srv1931777206=15, srv321476406=19, srv554946772=21, srv678078786=24, srv170805063=10, srv1828179758=12, srv1990144017=16, srv2013578581=17, srv136138055=6, srv474108304=20} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-15 15:57:38,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-15 15:57:38,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-15 15:57:38,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-15 15:57:38,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-15 15:57:38,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-15 15:57:38,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-15 15:57:38,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-15 15:57:38,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-15 15:57:38,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-15 15:57:38,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-15 15:57:38,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-15 15:57:38,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-15 15:57:38,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-15 15:57:38,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-15 15:57:38,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-15 15:57:38,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-15 15:57:38,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-15 15:57:38,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-15 15:57:38,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-15 15:57:38,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-15 15:57:38,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-15 15:57:38,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-15 15:57:38,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-15 15:57:38,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-15 15:57:38,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-15 15:57:38,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-15 15:57:38,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-15 15:57:38,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-15 15:57:38,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-15 15:57:38,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-15 15:57:38,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-04-15 15:57:38,973 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-15 15:57:38,973 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1043398963:90 , srv1124397980:90 , srv1155865125:90 , srv1177656639:90 , srv1331224046:90 , srv135214300:90 , srv136138055:90 , srv1619111420:90 , srv1632622053:90 , srv1687314384:90 , srv170805063:90 , srv1790406042:90 , srv1828179758:90 , srv1896543746:90 , srv1929261082:90 , srv1931777206:90 , srv1990144017:90 , srv2013578581:90 , srv2036600168:90 , srv321476406:90 , srv474108304:90 , srv554946772:90 , srv65065163:90 , srv675586104:90 , srv678078786:90 , srv721206205:90 , srv723654571:90 , srv785840451:90 , srv811688280:90 , srv876787162:90 }
2023-04-15 15:57:38,985 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=337 (was 384), ProcessCount=169 (was 169), AvailableMemoryMB=1092 (was 886) - AvailableMemoryMB LEAK? -