2023-09-29 10:58:39,066 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-09-29 10:58:39,296 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-09-29 10:58:39,382 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-09-29 10:58:39,431 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-09-29 10:58:39,437 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-09-29 10:58:39,447 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=186, ProcessCount=167, AvailableMemoryMB=7108
2023-09-29 10:58:39,449 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-09-29 10:58:39,449 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-09-29 10:58:39,554 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1011711923:84 , srv1014352866:84 , srv1029098772:84 , srv103141714:84 , srv1043509473:84 , srv1050950357:84 , srv1099173926:84 , srv1103385330:84 , srv1143767392:84 , srv1144736399:84 , srv1161095111:84 , srv1181654336:84 , srv1201476430:84 , srv1216698221:84 , srv1218625822:84 , srv124278502:84 , srv1261913925:84 , srv1274629058:84 , srv129221823:84 , srv1292402379:84 , srv1299234935:84 , srv1313356503:84 , srv1347898839:84 , srv1369516379:84 , srv1386417936:84 , srv1396442644:84 , srv1405323593:84 , srv1418327663:84 , srv1461205140:84 , srv1467192067:84 , srv1482506841:84 , srv1484585466:84 , srv1501445033:84 , srv1536639000:84 , srv1536976240:84 , srv1544746646:84 , srv1564115068:84 , srv1603240006:84 , srv1616655029:84 , srv1631805552:84 , srv1668988534:84 , srv172330090:84 , srv1740367393:84 , srv1741949291:84 , srv1802119230:84 , srv1803778969:84 , srv1854384908:84 , srv1858130905:84 , srv1903399850:84 , srv1906560271:84 , srv1916170987:84 , srv194550866:84 , srv1967295025:84 , srv1977046090:84 , srv2058742182:84 , srv2126905056:84 , srv22708205:84 , srv260442046:84 , srv260814034:84 , srv291337201:84 , srv2969595:84 , srv32041777:84 , srv330841531:84 , srv372400357:84 , srv387621028:84 , srv397480869:84 , srv422119917:84 , srv431465680:84 , srv435088572:84 , srv457095460:84 , srv460430682:84 , srv467760404:84 , srv476827942:84 , srv490099392:84 , srv529373890:84 , srv55566005:84 , srv614153850:84 , srv653153023:84 , srv658376944:84 , srv688119593:84 , srv737176701:84 , srv753056380:84 , srv781726526:84 , srv793807132:84 , srv799039332:84 , srv807198478:84 , srv84148399:84 , srv847315043:84 , srv847378486:84 , srv85140133:84 , srv859256537:84 , srv878837371:84 , srv887606392:84 , srv909307048:84 , srv960820705:84 , srv962759646:84 , srv974196056:84 , srv982515194:84 , srv983331950:84 , srv427331868:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-09-29 10:58:39,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-29 10:58:39,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv422119917=66, srv1536639000=33, srv129221823=18, srv962759646=96, srv32041777=61, srv1014352866=1, srv1482506841=30, srv1564115068=36, srv847378486=89, srv2969595=60, srv260442046=57, srv887606392=93, srv1467192067=29, srv1161095111=10, srv55566005=76, srv1802119230=44, srv859256537=91, srv1313356503=21, srv260814034=58, srv1418327663=27, srv476827942=73, srv1858130905=47, srv1218625822=14, srv1603240006=37, srv847315043=88, srv1216698221=13, srv1405323593=26, srv460430682=71, srv84148399=87, srv1668988534=40, srv960820705=95, srv737176701=81, srv1292402379=19, srv1967295025=52, srv103141714=3, srv1631805552=39, srv658376944=79, srv1261913925=16, srv85140133=90, srv688119593=80, srv1741949291=43, srv22708205=56, srv1903399850=48, srv614153850=77, srv1396442644=25, srv1299234935=20, srv1347898839=22, srv1616655029=38, srv1740367393=42, srv1906560271=49, srv1544746646=35, srv909307048=94, srv330841531=62, srv435088572=69, srv1144736399=9, srv1916170987=50, srv753056380=82, srv1029098772=2, srv124278502=15, srv397480869=65, srv194550866=51, srv1011711923=0, srv2126905056=55, srv1099173926=6, srv1103385330=7, srv291337201=59, srv653153023=78, srv1536976240=34, srv467760404=72, srv1501445033=32, srv1201476430=12, srv1369516379=23, srv983331950=99, srv529373890=75, srv781726526=83, srv799039332=85, srv1274629058=17, srv1461205140=28, srv982515194=98, srv878837371=92, srv1484585466=31, srv1050950357=5, srv372400357=63, srv387621028=64, srv490099392=74, srv793807132=84, srv807198478=86, srv1854384908=46, srv427331868=67, srv172330090=41, srv1181654336=11, srv1386417936=24, srv2058742182=54, srv431465680=68, srv1043509473=4, srv1977046090=53, srv1143767392=8, srv974196056=97, srv1803778969=45, srv457095460=70} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-29 10:58:39,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-29 10:58:39,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-29 10:58:39,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-29 10:58:39,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-29 10:58:39,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-29 10:58:39,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-29 10:58:39,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-29 10:58:39,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-29 10:58:39,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-29 10:58:39,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-29 10:58:39,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-29 10:58:39,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-29 10:58:39,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-29 10:58:39,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-29 10:58:39,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-29 10:58:39,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-29 10:58:39,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-29 10:58:39,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-29 10:58:39,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-29 10:58:39,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-29 10:58:39,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-29 10:58:39,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-29 10:58:39,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-29 10:58:39,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-29 10:58:39,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-29 10:58:39,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-29 10:58:39,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-29 10:58:39,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-29 10:58:39,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-29 10:58:39,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-29 10:58:39,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-29 10:58:39,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-29 10:58:39,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-29 10:58:39,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-29 10:58:39,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-29 10:58:39,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-29 10:58:39,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-29 10:58:39,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-29 10:58:39,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-29 10:58:39,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-29 10:58:39,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-29 10:58:39,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-29 10:58:39,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-29 10:58:39,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-29 10:58:39,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-29 10:58:39,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-29 10:58:39,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-29 10:58:39,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-29 10:58:39,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-29 10:58:39,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-29 10:58:39,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-29 10:58:39,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-29 10:58:39,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-29 10:58:39,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-29 10:58:39,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-29 10:58:39,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-29 10:58:39,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-29 10:58:39,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-29 10:58:39,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-29 10:58:39,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-29 10:58:39,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-29 10:58:39,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-29 10:58:39,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-29 10:58:39,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-29 10:58:39,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-29 10:58:39,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-29 10:58:39,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-29 10:58:39,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-29 10:58:39,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-29 10:58:39,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-29 10:58:39,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-29 10:58:39,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-29 10:58:39,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-29 10:58:39,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-29 10:58:39,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-29 10:58:39,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-29 10:58:39,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-29 10:58:39,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-29 10:58:39,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-29 10:58:39,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-29 10:58:39,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-29 10:58:39,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-29 10:58:39,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-29 10:58:39,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-29 10:58:39,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-29 10:58:39,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-29 10:58:39,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-29 10:58:39,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-29 10:58:39,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-29 10:58:39,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-29 10:58:39,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-29 10:58:39,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-29 10:58:39,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-29 10:58:39,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-29 10:58:39,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-29 10:58:39,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-29 10:58:39,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-29 10:58:39,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-29 10:58:39,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-29 10:58:39,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-29 10:58:39,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-29 10:58:39,894 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-29 10:58:39,894 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-29 10:58:39,894 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-29 10:58:40,277 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 612 ms to try 5000 different iterations.  Found a solution that moves 4661 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1813494840035255. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040258858128524894, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.055519180010569456, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5178888888888888, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040258858128524894, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17675, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24158333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-29 10:58:40,294 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1201476430:82 , srv1274629058:82 , srv1906560271:82 , srv431465680:82 , srv460430682:82 , srv1161095111:83 , srv1668988534:83 , srv422119917:83 , srv476827942:83 , srv847315043:83 , srv1029098772:84 , srv1144736399:84 , srv1501445033:84 , srv1536639000:84 , srv1631805552:84 , srv1854384908:84 , srv2058742182:84 , srv435088572:84 , srv457095460:84 , srv490099392:84 , srv737176701:84 , srv1050950357:85 , srv1103385330:85 , srv129221823:85 , srv1405323593:85 , srv1740367393:85 , srv194550866:85 , srv1967295025:85 , srv1977046090:85 , srv22708205:85 , srv260814034:85 , srv529373890:85 , srv614153850:85 , srv653153023:85 , srv793807132:85 , srv960820705:85 , srv103141714:86 , srv1143767392:86 , srv1313356503:86 , srv1467192067:86 , srv1536976240:86 , srv172330090:86 , srv1858130905:86 , srv1916170987:86 , srv2969595:86 , srv330841531:86 , srv397480869:86 , srv84148399:86 , srv847378486:86 , srv85140133:86 , srv887606392:86 , srv974196056:86 , srv124278502:87 , srv1261913925:87 , srv1299234935:87 , srv1369516379:87 , srv1461205140:87 , srv1482506841:87 , srv1603240006:87 , srv2126905056:87 , srv260442046:87 , srv32041777:87 , srv387621028:87 , srv467760404:87 , srv658376944:87 , srv781726526:87 , srv799039332:87 , srv859256537:87 , srv909307048:87 , srv983331950:87 , srv1011711923:88 , srv1043509473:88 , srv1216698221:88 , srv1347898839:88 , srv1418327663:88 , srv1544746646:88 , srv1564115068:88 , srv1802119230:88 , srv291337201:88 , srv982515194:88 , srv1099173926:89 , srv1218625822:89 , srv1396442644:89 , srv1616655029:89 , srv1741949291:89 , srv1803778969:89 , srv372400357:89 , srv807198478:89 , srv1014352866:90 , srv1386417936:90 , srv1484585466:90 , srv55566005:90 , srv753056380:90 , srv878837371:90 , srv962759646:90 , srv1181654336:91 , srv1292402379:91 , srv1903399850:91 , srv688119593:91 , srv427331868:448 }
2023-09-29 10:58:40,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-29 10:58:40,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv422119917=66, srv1536639000=33, srv129221823=18, srv962759646=96, srv32041777=61, srv1014352866=1, srv1482506841=30, srv1564115068=36, srv847378486=89, srv2969595=60, srv260442046=57, srv887606392=93, srv1467192067=29, srv1161095111=10, srv55566005=76, srv1802119230=44, srv859256537=91, srv1313356503=21, srv260814034=58, srv1418327663=27, srv476827942=73, srv1858130905=47, srv1218625822=14, srv1603240006=37, srv847315043=88, srv1216698221=13, srv1405323593=26, srv460430682=71, srv84148399=87, srv1668988534=40, srv960820705=95, srv737176701=81, srv1292402379=19, srv1967295025=52, srv103141714=3, srv1631805552=39, srv658376944=79, srv1261913925=16, srv85140133=90, srv688119593=80, srv1741949291=43, srv22708205=56, srv1903399850=48, srv614153850=77, srv1396442644=25, srv1299234935=20, srv1347898839=22, srv1616655029=38, srv1740367393=42, srv1906560271=49, srv1544746646=35, srv909307048=94, srv330841531=62, srv435088572=69, srv1144736399=9, srv1916170987=50, srv753056380=82, srv1029098772=2, srv124278502=15, srv397480869=65, srv194550866=51, srv1011711923=0, srv2126905056=55, srv1099173926=6, srv1103385330=7, srv291337201=59, srv653153023=78, srv1536976240=34, srv467760404=72, srv1501445033=32, srv1201476430=12, srv1369516379=23, srv983331950=99, srv529373890=75, srv781726526=83, srv799039332=85, srv1274629058=17, srv1461205140=28, srv982515194=98, srv878837371=92, srv1484585466=31, srv1050950357=5, srv372400357=63, srv387621028=64, srv490099392=74, srv793807132=84, srv807198478=86, srv1854384908=46, srv427331868=67, srv172330090=41, srv1181654336=11, srv1386417936=24, srv2058742182=54, srv431465680=68, srv1043509473=4, srv1977046090=53, srv1143767392=8, srv974196056=97, srv1803778969=45, srv457095460=70} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-29 10:58:40,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-29 10:58:40,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-29 10:58:40,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-29 10:58:40,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-29 10:58:40,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-29 10:58:40,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-29 10:58:40,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-29 10:58:40,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-29 10:58:40,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-29 10:58:40,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-29 10:58:40,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-29 10:58:40,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-29 10:58:40,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-29 10:58:40,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-29 10:58:40,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-29 10:58:40,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-29 10:58:40,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-29 10:58:40,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-29 10:58:40,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-29 10:58:40,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-29 10:58:40,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-29 10:58:40,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-29 10:58:40,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-29 10:58:40,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-29 10:58:40,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-29 10:58:40,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-29 10:58:40,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-29 10:58:40,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-29 10:58:40,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-29 10:58:40,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-29 10:58:40,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-29 10:58:40,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-29 10:58:40,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-29 10:58:40,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-29 10:58:40,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-29 10:58:40,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-29 10:58:40,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-29 10:58:40,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-29 10:58:40,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-29 10:58:40,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-29 10:58:40,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-29 10:58:40,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-29 10:58:40,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-29 10:58:40,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-29 10:58:40,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-29 10:58:40,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-29 10:58:40,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-29 10:58:40,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-29 10:58:40,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-29 10:58:40,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-29 10:58:40,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-29 10:58:40,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-29 10:58:40,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-29 10:58:40,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-29 10:58:40,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-29 10:58:40,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-29 10:58:40,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-29 10:58:40,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-29 10:58:40,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-29 10:58:40,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-29 10:58:40,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-29 10:58:40,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-29 10:58:40,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-29 10:58:40,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-29 10:58:40,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-29 10:58:40,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-29 10:58:40,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-29 10:58:40,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-29 10:58:40,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-29 10:58:40,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-29 10:58:40,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-29 10:58:40,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-29 10:58:40,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-29 10:58:40,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-29 10:58:40,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-29 10:58:40,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-29 10:58:40,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-29 10:58:40,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-29 10:58:40,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-29 10:58:40,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-29 10:58:40,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-29 10:58:40,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-29 10:58:40,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-29 10:58:40,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-29 10:58:40,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-29 10:58:40,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-29 10:58:40,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-29 10:58:40,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-29 10:58:40,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-29 10:58:40,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-29 10:58:40,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-29 10:58:40,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-29 10:58:40,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-29 10:58:40,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-29 10:58:40,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-29 10:58:40,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-29 10:58:40,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-29 10:58:40,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-29 10:58:40,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-29 10:58:40,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-29 10:58:40,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-29 10:58:40,433 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040258858128524894, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.055519180010569456, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040258858128524894, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17675, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24158333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-29 10:58:40,433 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-29 10:58:40,433 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18131684698396047, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040258858128524894, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.055519180010569456, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040258858128524894, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17675, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24158333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-29 10:58:40,632 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 289 ms to try 5000 different iterations.  Found a solution that moves 2618 regions; Going from a computed imbalance of 0.18131684698396047 to a new imbalance of 0.014548880995962809. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.024070706835219353); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03393067468565896, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29088888888888886, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.024070706835219353); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0115); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043416666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-29 10:58:40,639 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1292402379:80 , srv1396442644:80 , srv1903399850:80 , srv1014352866:81 , srv1181654336:81 , srv1218625822:81 , srv1386417936:81 , srv1484585466:81 , srv1803778969:81 , srv55566005:81 , srv753056380:81 , srv1564115068:82 , srv1616655029:82 , srv372400357:82 , srv688119593:82 , srv1347898839:83 , srv1418327663:83 , srv1461205140:83 , srv1467192067:83 , srv260442046:83 , srv291337201:83 , srv2969595:83 , srv878837371:83 , srv1099173926:84 , srv124278502:84 , srv1369516379:84 , srv1741949291:84 , srv387621028:84 , srv974196056:84 , srv983331950:84 , srv1050950357:85 , srv1216698221:85 , srv1299234935:85 , srv1544746646:85 , srv1603240006:85 , srv172330090:85 , srv32041777:85 , srv799039332:85 , srv807198478:85 , srv84148399:85 , srv859256537:85 , srv909307048:85 , srv962759646:85 , srv103141714:86 , srv1043509473:86 , srv1103385330:86 , srv1143767392:86 , srv1261913925:86 , srv1802119230:86 , srv1858130905:86 , srv1916170987:86 , srv1977046090:86 , srv2126905056:86 , srv260814034:86 , srv397480869:86 , srv457095460:86 , srv467760404:86 , srv529373890:86 , srv737176701:86 , srv781726526:86 , srv793807132:86 , srv847378486:86 , srv85140133:86 , srv960820705:86 , srv982515194:86 , srv1029098772:87 , srv1144736399:87 , srv1313356503:87 , srv1482506841:87 , srv1668988534:87 , srv1740367393:87 , srv194550866:87 , srv1967295025:87 , srv2058742182:87 , srv330841531:87 , srv435088572:87 , srv614153850:87 , srv847315043:87 , srv887606392:87 , srv1011711923:88 , srv1501445033:88 , srv1536639000:88 , srv1631805552:88 , srv22708205:88 , srv460430682:88 , srv653153023:88 , srv658376944:88 , srv1161095111:89 , srv1201476430:89 , srv129221823:89 , srv1405323593:89 , srv1536976240:89 , srv1906560271:89 , srv422119917:89 , srv431465680:89 , srv490099392:89 , srv1274629058:90 , srv1854384908:91 , srv476827942:91 , srv427331868:540 }
2023-09-29 10:58:40,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-29 10:58:40,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv422119917=66, srv1536639000=33, srv129221823=18, srv962759646=96, srv32041777=61, srv1014352866=1, srv1482506841=30, srv1564115068=36, srv847378486=89, srv2969595=60, srv260442046=57, srv887606392=93, srv1467192067=29, srv1161095111=10, srv55566005=76, srv1802119230=44, srv859256537=91, srv1313356503=21, srv260814034=58, srv1418327663=27, srv476827942=73, srv1858130905=47, srv1218625822=14, srv1603240006=37, srv847315043=88, srv1216698221=13, srv1405323593=26, srv460430682=71, srv84148399=87, srv1668988534=40, srv960820705=95, srv737176701=81, srv1292402379=19, srv1967295025=52, srv103141714=3, srv1631805552=39, srv658376944=79, srv1261913925=16, srv85140133=90, srv688119593=80, srv1741949291=43, srv22708205=56, srv1903399850=48, srv614153850=77, srv1396442644=25, srv1299234935=20, srv1347898839=22, srv1616655029=38, srv1740367393=42, srv1906560271=49, srv1544746646=35, srv909307048=94, srv330841531=62, srv435088572=69, srv1144736399=9, srv1916170987=50, srv753056380=82, srv1029098772=2, srv124278502=15, srv397480869=65, srv194550866=51, srv1011711923=0, srv2126905056=55, srv1099173926=6, srv1103385330=7, srv291337201=59, srv653153023=78, srv1536976240=34, srv467760404=72, srv1501445033=32, srv1201476430=12, srv1369516379=23, srv983331950=99, srv529373890=75, srv781726526=83, srv799039332=85, srv1274629058=17, srv1461205140=28, srv982515194=98, srv878837371=92, srv1484585466=31, srv1050950357=5, srv372400357=63, srv387621028=64, srv490099392=74, srv793807132=84, srv807198478=86, srv1854384908=46, srv427331868=67, srv172330090=41, srv1181654336=11, srv1386417936=24, srv2058742182=54, srv431465680=68, srv1043509473=4, srv1977046090=53, srv1143767392=8, srv974196056=97, srv1803778969=45, srv457095460=70} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-29 10:58:40,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-29 10:58:40,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-29 10:58:40,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-29 10:58:40,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-29 10:58:40,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-29 10:58:40,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-29 10:58:40,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-29 10:58:40,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-29 10:58:40,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-29 10:58:40,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-29 10:58:40,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-29 10:58:40,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-29 10:58:40,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-29 10:58:40,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-29 10:58:40,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-29 10:58:40,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-29 10:58:40,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-29 10:58:40,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-29 10:58:40,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-29 10:58:40,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-29 10:58:40,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-29 10:58:40,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-29 10:58:40,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-29 10:58:40,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-29 10:58:40,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-29 10:58:40,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-29 10:58:40,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-29 10:58:40,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-29 10:58:40,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-29 10:58:40,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-29 10:58:40,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-29 10:58:40,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-29 10:58:40,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-29 10:58:40,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-29 10:58:40,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-29 10:58:40,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-29 10:58:40,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-29 10:58:40,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-29 10:58:40,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-29 10:58:40,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-29 10:58:40,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-29 10:58:40,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-29 10:58:40,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-29 10:58:40,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-29 10:58:40,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-29 10:58:40,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-29 10:58:40,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-29 10:58:40,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-29 10:58:40,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-29 10:58:40,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-29 10:58:40,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-29 10:58:40,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-29 10:58:40,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-29 10:58:40,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-29 10:58:40,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-29 10:58:40,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-29 10:58:40,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-29 10:58:40,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-29 10:58:40,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-29 10:58:40,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-29 10:58:40,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-29 10:58:40,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-29 10:58:40,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-29 10:58:40,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-29 10:58:40,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-29 10:58:40,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-29 10:58:40,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-29 10:58:40,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-29 10:58:40,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-29 10:58:40,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-29 10:58:40,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-29 10:58:40,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-29 10:58:40,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-29 10:58:40,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-29 10:58:40,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-29 10:58:40,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-29 10:58:40,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-29 10:58:40,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-29 10:58:40,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-29 10:58:40,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-29 10:58:40,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-29 10:58:40,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-29 10:58:40,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-29 10:58:40,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-29 10:58:40,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-29 10:58:40,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-29 10:58:40,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-29 10:58:40,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-29 10:58:40,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-29 10:58:40,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-29 10:58:40,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-29 10:58:40,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-29 10:58:40,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-29 10:58:40,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-29 10:58:40,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-29 10:58:40,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-29 10:58:40,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-29 10:58:40,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-29 10:58:40,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-29 10:58:40,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-29 10:58:40,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-29 10:58:40,751 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.024070706835219353); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03393067468565896, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.024070706835219353); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0115); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043416666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-29 10:58:40,751 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-29 10:58:40,751 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.01453054936815307, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.024070706835219353); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03393067468565896, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.024070706835219353); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0115); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043416666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-29 10:58:40,910 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 234 ms to try 5000 different iterations.  Found a solution that moves 1011 regions; Going from a computed imbalance of 0.01453054936815307 to a new imbalance of 0.001078996099518806. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008356682589892833); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.010855649121315493); MoveCostFunction : (multiplier=7.0, imbalance=0.11233333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.008356682589892833); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=7.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-29 10:58:40,911 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1011711923:81 , srv129221823:82 , srv476827942:82 , srv658376944:82 , srv1043509473:83 , srv1216698221:83 , srv1405323593:83 , srv1482506841:83 , srv1536976240:83 , srv1802119230:83 , srv1854384908:83 , srv1099173926:84 , srv1143767392:84 , srv1261913925:84 , srv1274629058:84 , srv1544746646:84 , srv1741949291:84 , srv2126905056:84 , srv330841531:84 , srv467760404:84 , srv653153023:84 , srv85140133:84 , srv909307048:84 , srv962759646:84 , srv982515194:84 , srv103141714:85 , srv1181654336:85 , srv1292402379:85 , srv1313356503:85 , srv1386417936:85 , srv1484585466:85 , srv1536639000:85 , srv1616655029:85 , srv1740367393:85 , srv1858130905:85 , srv1906560271:85 , srv1916170987:85 , srv194550866:85 , srv1967295025:85 , srv1977046090:85 , srv2058742182:85 , srv22708205:85 , srv32041777:85 , srv422119917:85 , srv688119593:85 , srv781726526:85 , srv799039332:85 , srv807198478:85 , srv859256537:85 , srv878837371:85 , srv887606392:85 , srv1014352866:86 , srv1029098772:86 , srv1103385330:86 , srv1161095111:86 , srv1299234935:86 , srv1369516379:86 , srv1418327663:86 , srv1501445033:86 , srv1564115068:86 , srv1603240006:86 , srv1631805552:86 , srv172330090:86 , srv260814034:86 , srv291337201:86 , srv372400357:86 , srv387621028:86 , srv397480869:86 , srv431465680:86 , srv490099392:86 , srv529373890:86 , srv55566005:86 , srv793807132:86 , srv84148399:86 , srv847315043:86 , srv847378486:86 , srv983331950:86 , srv1201476430:87 , srv124278502:87 , srv1347898839:87 , srv1461205140:87 , srv1668988534:87 , srv1803778969:87 , srv1903399850:87 , srv260442046:87 , srv435088572:87 , srv614153850:87 , srv753056380:87 , srv974196056:87 , srv1050950357:88 , srv1218625822:88 , srv1396442644:88 , srv1467192067:88 , srv2969595:88 , srv457095460:88 , srv737176701:88 , srv960820705:88 , srv1144736399:89 , srv460430682:89 , srv427331868:544 }
2023-09-29 10:58:40,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-29 10:58:40,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv422119917=66, srv1536639000=33, srv129221823=18, srv962759646=96, srv32041777=61, srv1014352866=1, srv1482506841=30, srv1564115068=36, srv847378486=89, srv2969595=60, srv260442046=57, srv887606392=93, srv1467192067=29, srv1161095111=10, srv55566005=76, srv1802119230=44, srv859256537=91, srv1313356503=21, srv260814034=58, srv1418327663=27, srv476827942=73, srv1858130905=47, srv1218625822=14, srv1603240006=37, srv847315043=88, srv1216698221=13, srv1405323593=26, srv460430682=71, srv84148399=87, srv1668988534=40, srv960820705=95, srv737176701=81, srv1292402379=19, srv1967295025=52, srv103141714=3, srv1631805552=39, srv658376944=79, srv1261913925=16, srv85140133=90, srv688119593=80, srv1741949291=43, srv22708205=56, srv1903399850=48, srv614153850=77, srv1396442644=25, srv1299234935=20, srv1347898839=22, srv1616655029=38, srv1740367393=42, srv1906560271=49, srv1544746646=35, srv909307048=94, srv330841531=62, srv435088572=69, srv1144736399=9, srv1916170987=50, srv753056380=82, srv1029098772=2, srv124278502=15, srv397480869=65, srv194550866=51, srv1011711923=0, srv2126905056=55, srv1099173926=6, srv1103385330=7, srv291337201=59, srv653153023=78, srv1536976240=34, srv467760404=72, srv1501445033=32, srv1201476430=12, srv1369516379=23, srv983331950=99, srv529373890=75, srv781726526=83, srv799039332=85, srv1274629058=17, srv1461205140=28, srv982515194=98, srv878837371=92, srv1484585466=31, srv1050950357=5, srv372400357=63, srv387621028=64, srv490099392=74, srv793807132=84, srv807198478=86, srv1854384908=46, srv427331868=67, srv172330090=41, srv1181654336=11, srv1386417936=24, srv2058742182=54, srv431465680=68, srv1043509473=4, srv1977046090=53, srv1143767392=8, srv974196056=97, srv1803778969=45, srv457095460=70} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-29 10:58:40,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-29 10:58:40,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-29 10:58:40,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-29 10:58:40,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-29 10:58:40,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-29 10:58:40,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-29 10:58:40,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-29 10:58:40,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-29 10:58:40,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-29 10:58:40,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-29 10:58:40,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-29 10:58:40,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-29 10:58:40,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-29 10:58:40,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-29 10:58:40,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-29 10:58:40,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-29 10:58:40,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-29 10:58:40,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-29 10:58:40,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-29 10:58:40,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-29 10:58:40,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-29 10:58:40,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-29 10:58:40,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-29 10:58:40,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-29 10:58:40,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-29 10:58:40,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-29 10:58:40,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-29 10:58:40,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-29 10:58:40,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-29 10:58:40,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-29 10:58:40,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-29 10:58:40,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-29 10:58:40,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-29 10:58:40,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-29 10:58:40,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-29 10:58:40,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-29 10:58:40,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-29 10:58:40,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-29 10:58:40,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-29 10:58:40,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-29 10:58:40,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-29 10:58:40,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-29 10:58:40,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-29 10:58:40,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-29 10:58:40,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-29 10:58:40,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-29 10:58:40,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-29 10:58:40,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-29 10:58:40,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-29 10:58:40,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-29 10:58:40,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-29 10:58:40,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-29 10:58:40,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-29 10:58:40,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-29 10:58:40,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-29 10:58:40,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-29 10:58:40,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-29 10:58:40,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-29 10:58:40,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-29 10:58:40,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-29 10:58:40,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-29 10:58:40,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-29 10:58:40,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-29 10:58:40,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-29 10:58:40,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-29 10:58:40,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-29 10:58:40,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-29 10:58:40,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-29 10:58:40,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-29 10:58:40,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-29 10:58:40,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-29 10:58:40,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-29 10:58:40,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-29 10:58:40,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-29 10:58:40,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-29 10:58:40,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-29 10:58:40,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-29 10:58:40,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-29 10:58:40,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-29 10:58:40,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-29 10:58:40,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-29 10:58:40,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-29 10:58:40,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-29 10:58:40,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-29 10:58:40,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-29 10:58:40,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-29 10:58:40,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-29 10:58:40,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-29 10:58:40,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-29 10:58:40,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-29 10:58:40,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-29 10:58:40,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-29 10:58:40,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-29 10:58:40,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-29 10:58:40,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-29 10:58:40,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-29 10:58:40,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-29 10:58:40,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-29 10:58:40,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-29 10:58:40,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-29 10:58:40,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-29 10:58:41,015 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008356682589892833); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.010855649121315493); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.008356682589892833); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=7.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-29 10:58:41,015 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-29 10:58:41,015 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.001071916926212601, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008356682589892833); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.010855649121315493); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.008356682589892833); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=7.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-29 10:58:41,227 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 287 ms to try 5000 different iterations.  Found a solution that moves 260 regions; Going from a computed imbalance of 0.001071916926212601 to a new imbalance of 1.8348879395611907E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014982219165849825); MoveCostFunction : (multiplier=7.0, imbalance=0.028888888888888888, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-29 10:58:41,228 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv960820705:82 , srv1144736399:83 , srv457095460:83 , srv460430682:83 , srv490099392:83 , srv614153850:83 , srv807198478:83 , srv962759646:83 , srv1050950357:84 , srv1161095111:84 , srv1201476430:84 , srv1218625822:84 , srv1299234935:84 , srv1313356503:84 , srv1347898839:84 , srv1369516379:84 , srv1396442644:84 , srv1501445033:84 , srv1536976240:84 , srv1603240006:84 , srv1631805552:84 , srv1854384908:84 , srv1903399850:84 , srv22708205:84 , srv2969595:84 , srv397480869:84 , srv435088572:84 , srv658376944:84 , srv688119593:84 , srv737176701:84 , srv753056380:84 , srv781726526:84 , srv847378486:84 , srv887606392:84 , srv982515194:84 , srv1011711923:85 , srv1014352866:85 , srv103141714:85 , srv1043509473:85 , srv1099173926:85 , srv1103385330:85 , srv1181654336:85 , srv124278502:85 , srv1261913925:85 , srv1292402379:85 , srv1405323593:85 , srv1418327663:85 , srv1461205140:85 , srv1467192067:85 , srv1482506841:85 , srv1536639000:85 , srv1544746646:85 , srv1668988534:85 , srv172330090:85 , srv1740367393:85 , srv1741949291:85 , srv1802119230:85 , srv1803778969:85 , srv1858130905:85 , srv1906560271:85 , srv1916170987:85 , srv194550866:85 , srv1977046090:85 , srv2126905056:85 , srv260442046:85 , srv260814034:85 , srv291337201:85 , srv32041777:85 , srv330841531:85 , srv372400357:85 , srv387621028:85 , srv422119917:85 , srv467760404:85 , srv529373890:85 , srv55566005:85 , srv653153023:85 , srv799039332:85 , srv84148399:85 , srv859256537:85 , srv878837371:85 , srv974196056:85 , srv983331950:85 , srv1029098772:86 , srv1143767392:86 , srv1216698221:86 , srv1274629058:86 , srv129221823:86 , srv1386417936:86 , srv1484585466:86 , srv1564115068:86 , srv1616655029:86 , srv1967295025:86 , srv2058742182:86 , srv431465680:86 , srv476827942:86 , srv793807132:86 , srv85140133:86 , srv909307048:86 , srv847315043:87 , srv427331868:611 }
2023-09-29 10:58:41,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-29 10:58:41,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv422119917=66, srv1536639000=33, srv129221823=18, srv962759646=96, srv32041777=61, srv1014352866=1, srv1482506841=30, srv1564115068=36, srv847378486=89, srv2969595=60, srv260442046=57, srv887606392=93, srv1467192067=29, srv1161095111=10, srv55566005=76, srv1802119230=44, srv859256537=91, srv1313356503=21, srv260814034=58, srv1418327663=27, srv476827942=73, srv1858130905=47, srv1218625822=14, srv1603240006=37, srv847315043=88, srv1216698221=13, srv1405323593=26, srv460430682=71, srv84148399=87, srv1668988534=40, srv960820705=95, srv737176701=81, srv1292402379=19, srv1967295025=52, srv103141714=3, srv1631805552=39, srv658376944=79, srv1261913925=16, srv85140133=90, srv688119593=80, srv1741949291=43, srv22708205=56, srv1903399850=48, srv614153850=77, srv1396442644=25, srv1299234935=20, srv1347898839=22, srv1616655029=38, srv1740367393=42, srv1906560271=49, srv1544746646=35, srv909307048=94, srv330841531=62, srv435088572=69, srv1144736399=9, srv1916170987=50, srv753056380=82, srv1029098772=2, srv124278502=15, srv397480869=65, srv194550866=51, srv1011711923=0, srv2126905056=55, srv1099173926=6, srv1103385330=7, srv291337201=59, srv653153023=78, srv1536976240=34, srv467760404=72, srv1501445033=32, srv1201476430=12, srv1369516379=23, srv983331950=99, srv529373890=75, srv781726526=83, srv799039332=85, srv1274629058=17, srv1461205140=28, srv982515194=98, srv878837371=92, srv1484585466=31, srv1050950357=5, srv372400357=63, srv387621028=64, srv490099392=74, srv793807132=84, srv807198478=86, srv1854384908=46, srv427331868=67, srv172330090=41, srv1181654336=11, srv1386417936=24, srv2058742182=54, srv431465680=68, srv1043509473=4, srv1977046090=53, srv1143767392=8, srv974196056=97, srv1803778969=45, srv457095460=70} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-29 10:58:41,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-29 10:58:41,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-29 10:58:41,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-29 10:58:41,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-29 10:58:41,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-29 10:58:41,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-29 10:58:41,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-29 10:58:41,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-29 10:58:41,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-29 10:58:41,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-29 10:58:41,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-29 10:58:41,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-29 10:58:41,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-29 10:58:41,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-29 10:58:41,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-29 10:58:41,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-29 10:58:41,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-29 10:58:41,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-29 10:58:41,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-29 10:58:41,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-29 10:58:41,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-29 10:58:41,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-29 10:58:41,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-29 10:58:41,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-29 10:58:41,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-29 10:58:41,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-29 10:58:41,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-29 10:58:41,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-29 10:58:41,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-29 10:58:41,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-29 10:58:41,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-29 10:58:41,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-29 10:58:41,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-29 10:58:41,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-29 10:58:41,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-29 10:58:41,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-29 10:58:41,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-29 10:58:41,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-29 10:58:41,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-29 10:58:41,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-29 10:58:41,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-29 10:58:41,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-29 10:58:41,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-29 10:58:41,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-29 10:58:41,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-29 10:58:41,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-29 10:58:41,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-29 10:58:41,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-29 10:58:41,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-29 10:58:41,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-29 10:58:41,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-29 10:58:41,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-29 10:58:41,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-29 10:58:41,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-29 10:58:41,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-29 10:58:41,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-29 10:58:41,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-29 10:58:41,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-29 10:58:41,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-29 10:58:41,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-29 10:58:41,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-29 10:58:41,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-29 10:58:41,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-29 10:58:41,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-29 10:58:41,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-29 10:58:41,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-29 10:58:41,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-29 10:58:41,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-29 10:58:41,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-29 10:58:41,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-29 10:58:41,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-29 10:58:41,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-29 10:58:41,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-29 10:58:41,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-29 10:58:41,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-29 10:58:41,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-29 10:58:41,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-29 10:58:41,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-29 10:58:41,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-29 10:58:41,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-29 10:58:41,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-29 10:58:41,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-29 10:58:41,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-29 10:58:41,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-29 10:58:41,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-29 10:58:41,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-29 10:58:41,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-29 10:58:41,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-29 10:58:41,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-29 10:58:41,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-29 10:58:41,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-29 10:58:41,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-29 10:58:41,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-29 10:58:41,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-29 10:58:41,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-29 10:58:41,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-29 10:58:41,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-29 10:58:41,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-29 10:58:41,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-29 10:58:41,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-29 10:58:41,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-29 10:58:41,326 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.6528320484026048E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014982219165849825); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-29 10:58:41,326 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv960820705:82 , srv1144736399:83 , srv457095460:83 , srv460430682:83 , srv490099392:83 , srv614153850:83 , srv807198478:83 , srv962759646:83 , srv1050950357:84 , srv1161095111:84 , srv1201476430:84 , srv1218625822:84 , srv1299234935:84 , srv1313356503:84 , srv1347898839:84 , srv1369516379:84 , srv1396442644:84 , srv1501445033:84 , srv1536976240:84 , srv1603240006:84 , srv1631805552:84 , srv1854384908:84 , srv1903399850:84 , srv22708205:84 , srv2969595:84 , srv397480869:84 , srv435088572:84 , srv658376944:84 , srv688119593:84 , srv737176701:84 , srv753056380:84 , srv781726526:84 , srv847378486:84 , srv887606392:84 , srv982515194:84 , srv1011711923:85 , srv1014352866:85 , srv103141714:85 , srv1043509473:85 , srv1099173926:85 , srv1103385330:85 , srv1181654336:85 , srv124278502:85 , srv1261913925:85 , srv1292402379:85 , srv1405323593:85 , srv1418327663:85 , srv1461205140:85 , srv1467192067:85 , srv1482506841:85 , srv1536639000:85 , srv1544746646:85 , srv1668988534:85 , srv172330090:85 , srv1740367393:85 , srv1741949291:85 , srv1802119230:85 , srv1803778969:85 , srv1858130905:85 , srv1906560271:85 , srv1916170987:85 , srv194550866:85 , srv1977046090:85 , srv2126905056:85 , srv260442046:85 , srv260814034:85 , srv291337201:85 , srv32041777:85 , srv330841531:85 , srv372400357:85 , srv387621028:85 , srv422119917:85 , srv467760404:85 , srv529373890:85 , srv55566005:85 , srv653153023:85 , srv799039332:85 , srv84148399:85 , srv859256537:85 , srv878837371:85 , srv974196056:85 , srv983331950:85 , srv1029098772:86 , srv1143767392:86 , srv1216698221:86 , srv1274629058:86 , srv129221823:86 , srv1386417936:86 , srv1484585466:86 , srv1564115068:86 , srv1616655029:86 , srv1967295025:86 , srv2058742182:86 , srv431465680:86 , srv476827942:86 , srv793807132:86 , srv85140133:86 , srv909307048:86 , srv847315043:87 , srv427331868:611 }
2023-09-29 10:58:41,352 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=186 (was 186), ProcessCount=167 (was 167), AvailableMemoryMB=6617 (was 7108)
2023-09-29 10:58:41,359 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=186, ProcessCount=167, AvailableMemoryMB=6617
2023-09-29 10:58:41,360 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-09-29 10:58:41,360 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-09-29 10:58:41,370 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1038076585:84 , srv1043933232:84 , srv1091975534:84 , srv110474859:84 , srv1181302663:84 , srv1251913484:84 , srv1333955962:84 , srv1337346254:84 , srv149289591:84 , srv1590668501:84 , srv160548566:84 , srv1647579732:84 , srv1957859740:84 , srv2064268401:84 , srv292643286:84 , srv346268485:84 , srv38087693:84 , srv442501147:84 , srv477561729:84 , srv484681819:84 , srv565489352:84 , srv568113728:84 , srv570023177:84 , srv593096717:84 , srv703816033:84 , srv721927124:84 , srv762777581:84 , srv807721982:84 , srv950891904:84 , srv1666380156:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-09-29 10:58:41,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-29 10:58:41,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1666380156=12, srv570023177=23, srv149289591=8, srv2064268401=14, srv568113728=22, srv721927124=26, srv1181302663=4, srv1957859740=13, srv346268485=16, srv1091975534=2, srv38087693=17, srv1590668501=9, srv703816033=25, srv484681819=20, srv1647579732=11, srv160548566=10, srv1337346254=7, srv110474859=3, srv292643286=15, srv442501147=18, srv1333955962=6, srv950891904=29, srv1251913484=5, srv1043933232=1, srv565489352=21, srv477561729=19, srv762777581=27, srv593096717=24, srv1038076585=0, srv807721982=28} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-29 10:58:41,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-29 10:58:41,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-29 10:58:41,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-29 10:58:41,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-29 10:58:41,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-29 10:58:41,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-29 10:58:41,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-29 10:58:41,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-29 10:58:41,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-29 10:58:41,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-29 10:58:41,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-29 10:58:41,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-29 10:58:41,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-29 10:58:41,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-29 10:58:41,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-29 10:58:41,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-29 10:58:41,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-29 10:58:41,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-29 10:58:41,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-29 10:58:41,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-29 10:58:41,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-29 10:58:41,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-29 10:58:41,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-29 10:58:41,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-29 10:58:41,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-29 10:58:41,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-29 10:58:41,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-29 10:58:41,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-29 10:58:41,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-29 10:58:41,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-29 10:58:41,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-09-29 10:58:41,396 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0858509480290988, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-29 10:58:41,396 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912463556798228, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0858509480290988, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-09-29 11:00:41,385 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 10103151 different iterations.  Found a solution that moves 1836 regions; Going from a computed imbalance of 0.9912463556798228 to a new imbalance of 4.2853155918867096E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.68, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-29 11:00:41,390 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1038076585:90 , srv1043933232:90 , srv1091975534:90 , srv110474859:90 , srv1181302663:90 , srv1251913484:90 , srv1333955962:90 , srv1337346254:90 , srv149289591:90 , srv1590668501:90 , srv160548566:90 , srv1647579732:90 , srv1666380156:90 , srv1957859740:90 , srv2064268401:90 , srv292643286:90 , srv346268485:90 , srv38087693:90 , srv442501147:90 , srv477561729:90 , srv484681819:90 , srv565489352:90 , srv568113728:90 , srv570023177:90 , srv593096717:90 , srv703816033:90 , srv721927124:90 , srv762777581:90 , srv807721982:90 , srv950891904:90 }
2023-09-29 11:00:41,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-29 11:00:41,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1666380156=12, srv570023177=23, srv149289591=8, srv2064268401=14, srv568113728=22, srv721927124=26, srv1181302663=4, srv1957859740=13, srv346268485=16, srv1091975534=2, srv38087693=17, srv1590668501=9, srv703816033=25, srv484681819=20, srv1647579732=11, srv160548566=10, srv1337346254=7, srv110474859=3, srv292643286=15, srv442501147=18, srv1333955962=6, srv950891904=29, srv1251913484=5, srv1043933232=1, srv565489352=21, srv477561729=19, srv762777581=27, srv593096717=24, srv1038076585=0, srv807721982=28} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-29 11:00:41,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-29 11:00:41,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-29 11:00:41,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-29 11:00:41,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-29 11:00:41,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-29 11:00:41,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-29 11:00:41,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-29 11:00:41,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-29 11:00:41,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-29 11:00:41,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-29 11:00:41,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-29 11:00:41,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-29 11:00:41,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-29 11:00:41,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-29 11:00:41,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-29 11:00:41,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-29 11:00:41,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-29 11:00:41,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-29 11:00:41,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-29 11:00:41,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-29 11:00:41,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-29 11:00:41,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-29 11:00:41,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-29 11:00:41,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-29 11:00:41,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-29 11:00:41,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-29 11:00:41,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-29 11:00:41,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-29 11:00:41,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-29 11:00:41,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-29 11:00:41,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-09-29 11:00:41,405 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-29 11:00:41,405 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1038076585:90 , srv1043933232:90 , srv1091975534:90 , srv110474859:90 , srv1181302663:90 , srv1251913484:90 , srv1333955962:90 , srv1337346254:90 , srv149289591:90 , srv1590668501:90 , srv160548566:90 , srv1647579732:90 , srv1666380156:90 , srv1957859740:90 , srv2064268401:90 , srv292643286:90 , srv346268485:90 , srv38087693:90 , srv442501147:90 , srv477561729:90 , srv484681819:90 , srv565489352:90 , srv568113728:90 , srv570023177:90 , srv593096717:90 , srv703816033:90 , srv721927124:90 , srv762777581:90 , srv807721982:90 , srv950891904:90 }
2023-09-29 11:00:41,415 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=126 (was 186), ProcessCount=164 (was 167), AvailableMemoryMB=7502 (was 6617) - AvailableMemoryMB LEAK? -