2024-04-29 20:57:19,556 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2024-04-29 20:57:19,844 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2024-04-29 20:57:19,952 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2024-04-29 20:57:20,015 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-04-29 20:57:20,021 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-04-29 20:57:20,035 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=289, MaxFileDescriptor=60000, SystemLoadAverage=227, ProcessCount=193, AvailableMemoryMB=1201
2024-04-29 20:57:20,037 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-04-29 20:57:20,038 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-04-29 20:57:20,163 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1011762074:84 , srv101414353:84 , srv1025746016:84 , srv1034548264:84 , srv1052671661:84 , srv1055777773:84 , srv1056537097:84 , srv1087152861:84 , srv1101219882:84 , srv1116838561:84 , srv1131986737:84 , srv1165155912:84 , srv1190571016:84 , srv1202975165:84 , srv1210414209:84 , srv1217506746:84 , srv1227273520:84 , srv1239379441:84 , srv1254599585:84 , srv1278571519:84 , srv1279431395:84 , srv1313161803:84 , srv1340182069:84 , srv1342729722:84 , srv1391952820:84 , srv1410001766:84 , srv145291149:84 , srv1460715039:84 , srv1476155780:84 , srv148412557:84 , srv1487126456:84 , srv1498334414:84 , srv1511412242:84 , srv1527121751:84 , srv1530310319:84 , srv1554353059:84 , srv1563827731:84 , srv1574393054:84 , srv1584911188:84 , srv1594394692:84 , srv1597327192:84 , srv1646296750:84 , srv1695791047:84 , srv1708292330:84 , srv1724285571:84 , srv1760602475:84 , srv177262131:84 , srv1778830227:84 , srv181542222:84 , srv1851439780:84 , srv1885727211:84 , srv1893264417:84 , srv1912708385:84 , srv1922445663:84 , srv1930072885:84 , srv1944906385:84 , srv2008634590:84 , srv2017276095:84 , srv2046310599:84 , srv2089316631:84 , srv2092078213:84 , srv2092165496:84 , srv2092684395:84 , srv2123284948:84 , srv224777915:84 , srv246403091:84 , srv254712458:84 , srv260128640:84 , srv304260658:84 , srv32232895:84 , srv379404136:84 , srv38482267:84 , srv392318122:84 , srv395337061:84 , srv397374002:84 , srv442816343:84 , srv468700632:84 , srv475362811:84 , srv489656494:84 , srv500847637:84 , srv50690895:84 , srv542107106:84 , srv545929022:84 , srv568268577:84 , srv605523544:84 , srv696062847:84 , srv699194962:84 , srv749921508:84 , srv784731111:84 , srv806551161:84 , srv807911211:84 , srv82185574:84 , srv831503806:84 , srv832929462:84 , srv85835611:84 , srv859340292:84 , srv933986155:84 , srv991839427:84 , srv99857140:84 , srv192671733:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2024-04-29 20:57:20,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-29 20:57:20,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1034548264=3, srv1116838561=9, srv1912708385=52, srv1460715039=27, srv1052671661=4, srv500847637=80, srv260128640=68, srv859340292=96, srv2046310599=59, srv1055777773=5, srv1391952820=24, srv1487126456=30, srv1584911188=38, srv2008634590=57, srv1087152861=7, srv1011762074=0, srv2123284948=64, srv1893264417=51, srv1202975165=13, srv395337061=74, srv1190571016=12, srv1851439780=49, srv807911211=91, srv2092165496=62, srv1340182069=22, srv1885727211=50, srv38482267=72, srv991839427=98, srv545929022=83, srv1056537097=6, srv1527121751=33, srv1760602475=45, srv2092078213=61, srv1101219882=8, srv699194962=87, srv468700632=77, srv192671733=54, srv1131986737=10, srv696062847=86, srv933986155=97, srv254712458=67, srv1597327192=40, srv1279431395=20, srv1239379441=17, srv1922445663=53, srv246403091=66, srv85835611=95, srv1594394692=39, srv1227273520=16, srv1511412242=32, srv1278571519=19, srv397374002=75, srv806551161=90, srv1217506746=15, srv82185574=92, srv1778830227=47, srv2092684395=63, srv1574393054=37, srv832929462=94, srv99857140=99, srv50690895=81, srv1410001766=25, srv542107106=82, srv1210414209=14, srv181542222=48, srv2017276095=58, srv379404136=71, srv568268577=84, srv1254599585=18, srv1025746016=2, srv1530310319=34, srv1724285571=44, srv1313161803=21, srv32232895=70, srv1944906385=56, srv784731111=89, srv392318122=73, srv177262131=46, srv145291149=26, srv1708292330=43, srv475362811=78, srv2089316631=60, srv831503806=93, srv224777915=65, srv1554353059=35, srv101414353=1, srv1476155780=28, srv489656494=79, srv442816343=76, srv749921508=88, srv1165155912=11, srv1342729722=23, srv1563827731=36, srv304260658=69, srv148412557=29, srv1498334414=31, srv605523544=85, srv1646296750=41, srv1930072885=55, srv1695791047=42} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-29 20:57:20,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-29 20:57:20,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-29 20:57:20,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-29 20:57:20,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-29 20:57:20,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-29 20:57:20,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-29 20:57:20,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-29 20:57:20,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-29 20:57:20,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-29 20:57:20,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-29 20:57:20,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-29 20:57:20,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-29 20:57:20,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-29 20:57:20,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-29 20:57:20,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-29 20:57:20,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-29 20:57:20,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-29 20:57:20,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-29 20:57:20,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-29 20:57:20,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-29 20:57:20,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-29 20:57:20,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-29 20:57:20,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-29 20:57:20,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-29 20:57:20,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-29 20:57:20,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-29 20:57:20,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-29 20:57:20,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-29 20:57:20,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-29 20:57:20,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-29 20:57:20,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-29 20:57:20,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-29 20:57:20,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-29 20:57:20,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-29 20:57:20,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-29 20:57:20,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-29 20:57:20,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-29 20:57:20,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-29 20:57:20,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-29 20:57:20,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-29 20:57:20,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-29 20:57:20,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-29 20:57:20,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-29 20:57:20,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-29 20:57:20,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-29 20:57:20,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-29 20:57:20,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-29 20:57:20,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-29 20:57:20,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-29 20:57:20,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-29 20:57:20,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-29 20:57:20,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-29 20:57:20,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-29 20:57:20,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-29 20:57:20,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-29 20:57:20,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-29 20:57:20,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-29 20:57:20,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-29 20:57:20,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-29 20:57:20,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-29 20:57:20,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-29 20:57:20,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-29 20:57:20,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-29 20:57:20,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-29 20:57:20,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-29 20:57:20,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-29 20:57:20,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-29 20:57:20,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-29 20:57:20,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-29 20:57:20,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-29 20:57:20,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-29 20:57:20,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-29 20:57:20,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-29 20:57:20,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-29 20:57:20,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-29 20:57:20,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-29 20:57:20,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-29 20:57:20,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-29 20:57:20,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-29 20:57:20,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-29 20:57:20,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-29 20:57:20,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-29 20:57:20,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-29 20:57:20,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-29 20:57:20,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-29 20:57:20,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-29 20:57:20,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-29 20:57:20,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-29 20:57:20,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-29 20:57:20,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-29 20:57:20,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-29 20:57:20,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-29 20:57:20,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-29 20:57:20,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-29 20:57:20,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-29 20:57:20,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-29 20:57:20,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-29 20:57:20,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-29 20:57:20,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-29 20:57:20,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-29 20:57:20,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-29 20:57:20,608 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-29 20:57:20,609 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-29 20:57:20,609 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-29 20:57:21,005 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 678 ms to try 5000 different iterations.  Found a solution that moves 4712 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17816196592154276. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.042081103884672955, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.057036818400035716, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5235555555555556, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.042081103884672955, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1735, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2385, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-29 20:57:21,014 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv749921508:80 , srv1594394692:81 , srv1034548264:82 , srv1052671661:82 , srv542107106:82 , srv545929022:82 , srv1011762074:83 , srv1165155912:83 , srv1227273520:83 , srv1460715039:83 , srv1476155780:83 , srv1893264417:83 , srv831503806:83 , srv85835611:83 , srv1279431395:84 , srv1342729722:84 , srv1527121751:84 , srv1563827731:84 , srv1708292330:84 , srv1760602475:84 , srv181542222:84 , srv1912708385:84 , srv2008634590:84 , srv2017276095:84 , srv304260658:84 , srv468700632:84 , srv500847637:84 , srv807911211:84 , srv991839427:84 , srv1087152861:85 , srv1101219882:85 , srv148412557:85 , srv177262131:85 , srv1851439780:85 , srv1885727211:85 , srv2046310599:85 , srv2092684395:85 , srv254712458:85 , srv395337061:85 , srv397374002:85 , srv489656494:85 , srv1202975165:86 , srv1217506746:86 , srv1313161803:86 , srv1340182069:86 , srv1410001766:86 , srv1498334414:86 , srv1695791047:86 , srv1778830227:86 , srv1930072885:86 , srv2089316631:86 , srv2123284948:86 , srv99857140:86 , srv1056537097:87 , srv1116838561:87 , srv1131986737:87 , srv1190571016:87 , srv1239379441:87 , srv1254599585:87 , srv1278571519:87 , srv1391952820:87 , srv1597327192:87 , srv1724285571:87 , srv2092165496:87 , srv224777915:87 , srv246403091:87 , srv260128640:87 , srv32232895:87 , srv568268577:87 , srv699194962:87 , srv784731111:87 , srv82185574:87 , srv1487126456:88 , srv1554353059:88 , srv1584911188:88 , srv1922445663:88 , srv2092078213:88 , srv442816343:88 , srv806551161:88 , srv933986155:88 , srv1025746016:89 , srv1055777773:89 , srv1210414209:89 , srv1511412242:89 , srv1574393054:89 , srv379404136:89 , srv38482267:89 , srv50690895:89 , srv605523544:89 , srv696062847:89 , srv1530310319:90 , srv1944906385:90 , srv832929462:90 , srv392318122:91 , srv475362811:91 , srv859340292:91 , srv1646296750:92 , srv145291149:93 , srv101414353:96 , srv192671733:464 }
2024-04-29 20:57:21,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-29 20:57:21,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1034548264=3, srv1116838561=9, srv1912708385=52, srv1460715039=27, srv1052671661=4, srv500847637=80, srv260128640=68, srv859340292=96, srv2046310599=59, srv1055777773=5, srv1391952820=24, srv1487126456=30, srv1584911188=38, srv2008634590=57, srv1087152861=7, srv1011762074=0, srv2123284948=64, srv1893264417=51, srv1202975165=13, srv395337061=74, srv1190571016=12, srv1851439780=49, srv807911211=91, srv2092165496=62, srv1340182069=22, srv1885727211=50, srv38482267=72, srv991839427=98, srv545929022=83, srv1056537097=6, srv1527121751=33, srv1760602475=45, srv2092078213=61, srv1101219882=8, srv699194962=87, srv468700632=77, srv192671733=54, srv1131986737=10, srv696062847=86, srv933986155=97, srv254712458=67, srv1597327192=40, srv1279431395=20, srv1239379441=17, srv1922445663=53, srv246403091=66, srv85835611=95, srv1594394692=39, srv1227273520=16, srv1511412242=32, srv1278571519=19, srv397374002=75, srv806551161=90, srv1217506746=15, srv82185574=92, srv1778830227=47, srv2092684395=63, srv1574393054=37, srv832929462=94, srv99857140=99, srv50690895=81, srv1410001766=25, srv542107106=82, srv1210414209=14, srv181542222=48, srv2017276095=58, srv379404136=71, srv568268577=84, srv1254599585=18, srv1025746016=2, srv1530310319=34, srv1724285571=44, srv1313161803=21, srv32232895=70, srv1944906385=56, srv784731111=89, srv392318122=73, srv177262131=46, srv145291149=26, srv1708292330=43, srv475362811=78, srv2089316631=60, srv831503806=93, srv224777915=65, srv1554353059=35, srv101414353=1, srv1476155780=28, srv489656494=79, srv442816343=76, srv749921508=88, srv1165155912=11, srv1342729722=23, srv1563827731=36, srv304260658=69, srv148412557=29, srv1498334414=31, srv605523544=85, srv1646296750=41, srv1930072885=55, srv1695791047=42} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-29 20:57:21,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-29 20:57:21,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-29 20:57:21,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-29 20:57:21,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-29 20:57:21,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-29 20:57:21,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-29 20:57:21,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-29 20:57:21,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-29 20:57:21,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-29 20:57:21,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-29 20:57:21,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-29 20:57:21,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-29 20:57:21,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-29 20:57:21,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-29 20:57:21,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-29 20:57:21,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-29 20:57:21,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-29 20:57:21,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-29 20:57:21,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-29 20:57:21,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-29 20:57:21,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-29 20:57:21,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-29 20:57:21,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-29 20:57:21,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-29 20:57:21,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-29 20:57:21,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-29 20:57:21,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-29 20:57:21,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-29 20:57:21,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-29 20:57:21,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-29 20:57:21,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-29 20:57:21,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-29 20:57:21,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-29 20:57:21,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-29 20:57:21,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-29 20:57:21,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-29 20:57:21,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-29 20:57:21,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-29 20:57:21,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-29 20:57:21,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-29 20:57:21,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-29 20:57:21,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-29 20:57:21,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-29 20:57:21,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-29 20:57:21,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-29 20:57:21,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-29 20:57:21,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-29 20:57:21,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-29 20:57:21,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-29 20:57:21,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-29 20:57:21,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-29 20:57:21,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-29 20:57:21,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-29 20:57:21,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-29 20:57:21,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-29 20:57:21,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-29 20:57:21,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-29 20:57:21,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-29 20:57:21,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-29 20:57:21,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-29 20:57:21,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-29 20:57:21,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-29 20:57:21,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-29 20:57:21,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-29 20:57:21,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-29 20:57:21,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-29 20:57:21,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-29 20:57:21,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-29 20:57:21,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-29 20:57:21,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-29 20:57:21,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-29 20:57:21,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-29 20:57:21,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-29 20:57:21,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-29 20:57:21,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-29 20:57:21,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-29 20:57:21,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-29 20:57:21,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-29 20:57:21,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-29 20:57:21,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-29 20:57:21,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-29 20:57:21,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-29 20:57:21,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-29 20:57:21,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-29 20:57:21,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-29 20:57:21,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-29 20:57:21,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-29 20:57:21,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-29 20:57:21,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-29 20:57:21,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-29 20:57:21,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-29 20:57:21,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-29 20:57:21,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-29 20:57:21,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-29 20:57:21,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-29 20:57:21,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-29 20:57:21,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-29 20:57:21,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-29 20:57:21,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-29 20:57:21,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-29 20:57:21,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-29 20:57:21,148 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.042081103884672955, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.057036818400035716, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.042081103884672955, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1735, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2385, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-29 20:57:21,148 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-29 20:57:21,148 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.1781289717923451, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.042081103884672955, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.057036818400035716, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.042081103884672955, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1735, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2385, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-29 20:57:21,358 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 302 ms to try 5000 different iterations.  Found a solution that moves 2597 regions; Going from a computed imbalance of 0.1781289717923451 to a new imbalance of 0.01565578835181216. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02450762165413114); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03606640589411381, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.28855555555555557, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02450762165413114); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01225); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04808333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-29 20:57:21,367 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv101414353:76 , srv145291149:79 , srv1646296750:80 , srv475362811:80 , srv1055777773:81 , srv1210414209:81 , srv1511412242:82 , srv1530310319:82 , srv1554353059:82 , srv379404136:82 , srv392318122:82 , srv50690895:82 , srv605523544:82 , srv696062847:82 , srv1278571519:83 , srv1597327192:83 , srv2092078213:83 , srv246403091:83 , srv32232895:83 , srv806551161:83 , srv832929462:83 , srv1025746016:84 , srv1056537097:84 , srv1190571016:84 , srv1202975165:84 , srv1574393054:84 , srv1922445663:84 , srv1944906385:84 , srv38482267:84 , srv442816343:84 , srv859340292:84 , srv1101219882:85 , srv1116838561:85 , srv1131986737:85 , srv1239379441:85 , srv1254599585:85 , srv1313161803:85 , srv1410001766:85 , srv1487126456:85 , srv1498334414:85 , srv1584911188:85 , srv1724285571:85 , srv1930072885:85 , srv2092165496:85 , srv489656494:85 , srv568268577:85 , srv699194962:85 , srv784731111:85 , srv82185574:85 , srv933986155:85 , srv1217506746:86 , srv1340182069:86 , srv1391952820:86 , srv1527121751:86 , srv1695791047:86 , srv177262131:86 , srv2046310599:86 , srv2123284948:86 , srv224777915:86 , srv260128640:86 , srv99857140:86 , srv1165155912:87 , srv1279431395:87 , srv1563827731:87 , srv1708292330:87 , srv1851439780:87 , srv1893264417:87 , srv1912708385:87 , srv2017276095:87 , srv254712458:87 , srv395337061:87 , srv397374002:87 , srv468700632:87 , srv831503806:87 , srv1011762074:88 , srv1227273520:88 , srv1460715039:88 , srv148412557:88 , srv1594394692:88 , srv1760602475:88 , srv1778830227:88 , srv181542222:88 , srv2092684395:88 , srv500847637:88 , srv542107106:88 , srv807911211:88 , srv991839427:88 , srv1087152861:89 , srv1885727211:89 , srv2089316631:89 , srv304260658:89 , srv85835611:89 , srv1052671661:90 , srv1342729722:90 , srv2008634590:90 , srv545929022:90 , srv1034548264:91 , srv1476155780:92 , srv749921508:94 , srv192671733:528 }
2024-04-29 20:57:21,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-29 20:57:21,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1034548264=3, srv1116838561=9, srv1912708385=52, srv1460715039=27, srv1052671661=4, srv500847637=80, srv260128640=68, srv859340292=96, srv2046310599=59, srv1055777773=5, srv1391952820=24, srv1487126456=30, srv1584911188=38, srv2008634590=57, srv1087152861=7, srv1011762074=0, srv2123284948=64, srv1893264417=51, srv1202975165=13, srv395337061=74, srv1190571016=12, srv1851439780=49, srv807911211=91, srv2092165496=62, srv1340182069=22, srv1885727211=50, srv38482267=72, srv991839427=98, srv545929022=83, srv1056537097=6, srv1527121751=33, srv1760602475=45, srv2092078213=61, srv1101219882=8, srv699194962=87, srv468700632=77, srv192671733=54, srv1131986737=10, srv696062847=86, srv933986155=97, srv254712458=67, srv1597327192=40, srv1279431395=20, srv1239379441=17, srv1922445663=53, srv246403091=66, srv85835611=95, srv1594394692=39, srv1227273520=16, srv1511412242=32, srv1278571519=19, srv397374002=75, srv806551161=90, srv1217506746=15, srv82185574=92, srv1778830227=47, srv2092684395=63, srv1574393054=37, srv832929462=94, srv99857140=99, srv50690895=81, srv1410001766=25, srv542107106=82, srv1210414209=14, srv181542222=48, srv2017276095=58, srv379404136=71, srv568268577=84, srv1254599585=18, srv1025746016=2, srv1530310319=34, srv1724285571=44, srv1313161803=21, srv32232895=70, srv1944906385=56, srv784731111=89, srv392318122=73, srv177262131=46, srv145291149=26, srv1708292330=43, srv475362811=78, srv2089316631=60, srv831503806=93, srv224777915=65, srv1554353059=35, srv101414353=1, srv1476155780=28, srv489656494=79, srv442816343=76, srv749921508=88, srv1165155912=11, srv1342729722=23, srv1563827731=36, srv304260658=69, srv148412557=29, srv1498334414=31, srv605523544=85, srv1646296750=41, srv1930072885=55, srv1695791047=42} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-29 20:57:21,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-29 20:57:21,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-29 20:57:21,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-29 20:57:21,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-29 20:57:21,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-29 20:57:21,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-29 20:57:21,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-29 20:57:21,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-29 20:57:21,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-29 20:57:21,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-29 20:57:21,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-29 20:57:21,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-29 20:57:21,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-29 20:57:21,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-29 20:57:21,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-29 20:57:21,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-29 20:57:21,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-29 20:57:21,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-29 20:57:21,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-29 20:57:21,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-29 20:57:21,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-29 20:57:21,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-29 20:57:21,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-29 20:57:21,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-29 20:57:21,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-29 20:57:21,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-29 20:57:21,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-29 20:57:21,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-29 20:57:21,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-29 20:57:21,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-29 20:57:21,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-29 20:57:21,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-29 20:57:21,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-29 20:57:21,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-29 20:57:21,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-29 20:57:21,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-29 20:57:21,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-29 20:57:21,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-29 20:57:21,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-29 20:57:21,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-29 20:57:21,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-29 20:57:21,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-29 20:57:21,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-29 20:57:21,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-29 20:57:21,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-29 20:57:21,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-29 20:57:21,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-29 20:57:21,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-29 20:57:21,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-29 20:57:21,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-29 20:57:21,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-29 20:57:21,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-29 20:57:21,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-29 20:57:21,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-29 20:57:21,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-29 20:57:21,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-29 20:57:21,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-29 20:57:21,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-29 20:57:21,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-29 20:57:21,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-29 20:57:21,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-29 20:57:21,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-29 20:57:21,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-29 20:57:21,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-29 20:57:21,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-29 20:57:21,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-29 20:57:21,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-29 20:57:21,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-29 20:57:21,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-29 20:57:21,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-29 20:57:21,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-29 20:57:21,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-29 20:57:21,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-29 20:57:21,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-29 20:57:21,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-29 20:57:21,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-29 20:57:21,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-29 20:57:21,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-29 20:57:21,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-29 20:57:21,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-29 20:57:21,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-29 20:57:21,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-29 20:57:21,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-29 20:57:21,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-29 20:57:21,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-29 20:57:21,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-29 20:57:21,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-29 20:57:21,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-29 20:57:21,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-29 20:57:21,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-29 20:57:21,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-29 20:57:21,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-29 20:57:21,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-29 20:57:21,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-29 20:57:21,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-29 20:57:21,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-29 20:57:21,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-29 20:57:21,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-29 20:57:21,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-29 20:57:21,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-29 20:57:21,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-29 20:57:21,512 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02450762165413114); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03606640589411381, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02450762165413114); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01225); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04808333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-29 20:57:21,512 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-29 20:57:21,513 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.01563760376914528, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02450762165413114); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03606640589411381, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02450762165413114); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01225); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04808333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-29 20:57:21,702 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 293 ms to try 5000 different iterations.  Found a solution that moves 985 regions; Going from a computed imbalance of 0.01563760376914528 to a new imbalance of 0.001239830861062483. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007562352524690594); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007476112351500336); MoveCostFunction : (multiplier=7.0, imbalance=0.10944444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007562352524690594); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333334E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004583333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-29 20:57:21,703 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv859340292:81 , srv1476155780:83 , srv148412557:83 , srv1885727211:83 , srv2008634590:83 , srv2089316631:83 , srv2092165496:83 , srv2092684395:83 , srv749921508:83 , srv1034548264:84 , srv1087152861:84 , srv1116838561:84 , srv1342729722:84 , srv1487126456:84 , srv1584911188:84 , srv1778830227:84 , srv181542222:84 , srv1944906385:84 , srv260128640:84 , srv933986155:84 , srv99857140:84 , srv1025746016:85 , srv1131986737:85 , srv1239379441:85 , srv1254599585:85 , srv1340182069:85 , srv1391952820:85 , srv145291149:85 , srv1530310319:85 , srv1574393054:85 , srv1646296750:85 , srv1695791047:85 , srv1724285571:85 , srv1760602475:85 , srv1851439780:85 , srv1922445663:85 , srv2123284948:85 , srv224777915:85 , srv254712458:85 , srv304260658:85 , srv38482267:85 , srv392318122:85 , srv395337061:85 , srv397374002:85 , srv442816343:85 , srv696062847:85 , srv806551161:85 , srv832929462:85 , srv85835611:85 , srv991839427:85 , srv101414353:86 , srv1052671661:86 , srv1056537097:86 , srv1217506746:86 , srv1313161803:86 , srv1410001766:86 , srv1460715039:86 , srv1511412242:86 , srv1708292330:86 , srv177262131:86 , srv1912708385:86 , srv1930072885:86 , srv2017276095:86 , srv2046310599:86 , srv2092078213:86 , srv468700632:86 , srv475362811:86 , srv500847637:86 , srv545929022:86 , srv568268577:86 , srv605523544:86 , srv784731111:86 , srv807911211:86 , srv82185574:86 , srv1011762074:87 , srv1055777773:87 , srv1101219882:87 , srv1165155912:87 , srv1227273520:87 , srv1278571519:87 , srv1279431395:87 , srv1498334414:87 , srv1527121751:87 , srv1554353059:87 , srv1563827731:87 , srv246403091:87 , srv379404136:87 , srv489656494:87 , srv542107106:87 , srv699194962:87 , srv831503806:87 , srv1190571016:88 , srv1202975165:88 , srv1210414209:88 , srv1597327192:88 , srv32232895:88 , srv50690895:88 , srv1594394692:89 , srv1893264417:89 , srv192671733:533 }
2024-04-29 20:57:21,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-29 20:57:21,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1034548264=3, srv1116838561=9, srv1912708385=52, srv1460715039=27, srv1052671661=4, srv500847637=80, srv260128640=68, srv859340292=96, srv2046310599=59, srv1055777773=5, srv1391952820=24, srv1487126456=30, srv1584911188=38, srv2008634590=57, srv1087152861=7, srv1011762074=0, srv2123284948=64, srv1893264417=51, srv1202975165=13, srv395337061=74, srv1190571016=12, srv1851439780=49, srv807911211=91, srv2092165496=62, srv1340182069=22, srv1885727211=50, srv38482267=72, srv991839427=98, srv545929022=83, srv1056537097=6, srv1527121751=33, srv1760602475=45, srv2092078213=61, srv1101219882=8, srv699194962=87, srv468700632=77, srv192671733=54, srv1131986737=10, srv696062847=86, srv933986155=97, srv254712458=67, srv1597327192=40, srv1279431395=20, srv1239379441=17, srv1922445663=53, srv246403091=66, srv85835611=95, srv1594394692=39, srv1227273520=16, srv1511412242=32, srv1278571519=19, srv397374002=75, srv806551161=90, srv1217506746=15, srv82185574=92, srv1778830227=47, srv2092684395=63, srv1574393054=37, srv832929462=94, srv99857140=99, srv50690895=81, srv1410001766=25, srv542107106=82, srv1210414209=14, srv181542222=48, srv2017276095=58, srv379404136=71, srv568268577=84, srv1254599585=18, srv1025746016=2, srv1530310319=34, srv1724285571=44, srv1313161803=21, srv32232895=70, srv1944906385=56, srv784731111=89, srv392318122=73, srv177262131=46, srv145291149=26, srv1708292330=43, srv475362811=78, srv2089316631=60, srv831503806=93, srv224777915=65, srv1554353059=35, srv101414353=1, srv1476155780=28, srv489656494=79, srv442816343=76, srv749921508=88, srv1165155912=11, srv1342729722=23, srv1563827731=36, srv304260658=69, srv148412557=29, srv1498334414=31, srv605523544=85, srv1646296750=41, srv1930072885=55, srv1695791047=42} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-29 20:57:21,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-29 20:57:21,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-29 20:57:21,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-29 20:57:21,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-29 20:57:21,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-29 20:57:21,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-29 20:57:21,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-29 20:57:21,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-29 20:57:21,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-29 20:57:21,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-29 20:57:21,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-29 20:57:21,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-29 20:57:21,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-29 20:57:21,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-29 20:57:21,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-29 20:57:21,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-29 20:57:21,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-29 20:57:21,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-29 20:57:21,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-29 20:57:21,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-29 20:57:21,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-29 20:57:21,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-29 20:57:21,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-29 20:57:21,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-29 20:57:21,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-29 20:57:21,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-29 20:57:21,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-29 20:57:21,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-29 20:57:21,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-29 20:57:21,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-29 20:57:21,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-29 20:57:21,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-29 20:57:21,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-29 20:57:21,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-29 20:57:21,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-29 20:57:21,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-29 20:57:21,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-29 20:57:21,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-29 20:57:21,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-29 20:57:21,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-29 20:57:21,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-29 20:57:21,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-29 20:57:21,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-29 20:57:21,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-29 20:57:21,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-29 20:57:21,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-29 20:57:21,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-29 20:57:21,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-29 20:57:21,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-29 20:57:21,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-29 20:57:21,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-29 20:57:21,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-29 20:57:21,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-29 20:57:21,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-29 20:57:21,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-29 20:57:21,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-29 20:57:21,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-29 20:57:21,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-29 20:57:21,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-29 20:57:21,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-29 20:57:21,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-29 20:57:21,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-29 20:57:21,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-29 20:57:21,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-29 20:57:21,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-29 20:57:21,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-29 20:57:21,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-29 20:57:21,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-29 20:57:21,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-29 20:57:21,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-29 20:57:21,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-29 20:57:21,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-29 20:57:21,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-29 20:57:21,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-29 20:57:21,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-29 20:57:21,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-29 20:57:21,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-29 20:57:21,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-29 20:57:21,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-29 20:57:21,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-29 20:57:21,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-29 20:57:21,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-29 20:57:21,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-29 20:57:21,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-29 20:57:21,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-29 20:57:21,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-29 20:57:21,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-29 20:57:21,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-29 20:57:21,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-29 20:57:21,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-29 20:57:21,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-29 20:57:21,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-29 20:57:21,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-29 20:57:21,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-29 20:57:21,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-29 20:57:21,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-29 20:57:21,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-29 20:57:21,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-29 20:57:21,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-29 20:57:21,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-29 20:57:21,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-29 20:57:21,826 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007562352524690594); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007476112351500336); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007562352524690594); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333334E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004583333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-29 20:57:21,826 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-29 20:57:21,826 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.0012329337436474365, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007562352524690594); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007476112351500336); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007562352524690594); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333334E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004583333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-29 20:57:22,072 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 323 ms to try 5000 different iterations.  Found a solution that moves 304 regions; Going from a computed imbalance of 0.0012329337436474365 to a new imbalance of 4.495261719591438E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0023688968483956715); MoveCostFunction : (multiplier=7.0, imbalance=0.033777777777777775, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-29 20:57:22,073 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1025746016:83 , srv1190571016:83 , srv50690895:83 , srv699194962:83 , srv807911211:83 , srv1011762074:84 , srv1052671661:84 , srv1087152861:84 , srv1202975165:84 , srv1210414209:84 , srv1217506746:84 , srv1227273520:84 , srv1254599585:84 , srv1279431395:84 , srv1342729722:84 , srv1391952820:84 , srv1460715039:84 , srv1476155780:84 , srv1487126456:84 , srv1498334414:84 , srv1563827731:84 , srv1574393054:84 , srv1594394692:84 , srv1597327192:84 , srv1760602475:84 , srv177262131:84 , srv1778830227:84 , srv1893264417:84 , srv1944906385:84 , srv2089316631:84 , srv224777915:84 , srv304260658:84 , srv32232895:84 , srv379404136:84 , srv38482267:84 , srv392318122:84 , srv395337061:84 , srv500847637:84 , srv545929022:84 , srv568268577:84 , srv784731111:84 , srv82185574:84 , srv832929462:84 , srv101414353:85 , srv1034548264:85 , srv1055777773:85 , srv1056537097:85 , srv1101219882:85 , srv1131986737:85 , srv1165155912:85 , srv1239379441:85 , srv1278571519:85 , srv1313161803:85 , srv1340182069:85 , srv145291149:85 , srv1511412242:85 , srv1527121751:85 , srv1530310319:85 , srv1554353059:85 , srv1584911188:85 , srv1646296750:85 , srv1695791047:85 , srv1708292330:85 , srv1724285571:85 , srv1885727211:85 , srv1912708385:85 , srv1930072885:85 , srv2008634590:85 , srv2017276095:85 , srv2046310599:85 , srv2123284948:85 , srv246403091:85 , srv254712458:85 , srv397374002:85 , srv442816343:85 , srv468700632:85 , srv475362811:85 , srv489656494:85 , srv542107106:85 , srv749921508:85 , srv831503806:85 , srv85835611:85 , srv933986155:85 , srv991839427:85 , srv1116838561:86 , srv1410001766:86 , srv148412557:86 , srv181542222:86 , srv1851439780:86 , srv1922445663:86 , srv2092078213:86 , srv2092684395:86 , srv260128640:86 , srv605523544:86 , srv696062847:86 , srv806551161:86 , srv859340292:86 , srv99857140:86 , srv2092165496:87 , srv192671733:617 }
2024-04-29 20:57:22,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-29 20:57:22,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1034548264=3, srv1116838561=9, srv1912708385=52, srv1460715039=27, srv1052671661=4, srv500847637=80, srv260128640=68, srv859340292=96, srv2046310599=59, srv1055777773=5, srv1391952820=24, srv1487126456=30, srv1584911188=38, srv2008634590=57, srv1087152861=7, srv1011762074=0, srv2123284948=64, srv1893264417=51, srv1202975165=13, srv395337061=74, srv1190571016=12, srv1851439780=49, srv807911211=91, srv2092165496=62, srv1340182069=22, srv1885727211=50, srv38482267=72, srv991839427=98, srv545929022=83, srv1056537097=6, srv1527121751=33, srv1760602475=45, srv2092078213=61, srv1101219882=8, srv699194962=87, srv468700632=77, srv192671733=54, srv1131986737=10, srv696062847=86, srv933986155=97, srv254712458=67, srv1597327192=40, srv1279431395=20, srv1239379441=17, srv1922445663=53, srv246403091=66, srv85835611=95, srv1594394692=39, srv1227273520=16, srv1511412242=32, srv1278571519=19, srv397374002=75, srv806551161=90, srv1217506746=15, srv82185574=92, srv1778830227=47, srv2092684395=63, srv1574393054=37, srv832929462=94, srv99857140=99, srv50690895=81, srv1410001766=25, srv542107106=82, srv1210414209=14, srv181542222=48, srv2017276095=58, srv379404136=71, srv568268577=84, srv1254599585=18, srv1025746016=2, srv1530310319=34, srv1724285571=44, srv1313161803=21, srv32232895=70, srv1944906385=56, srv784731111=89, srv392318122=73, srv177262131=46, srv145291149=26, srv1708292330=43, srv475362811=78, srv2089316631=60, srv831503806=93, srv224777915=65, srv1554353059=35, srv101414353=1, srv1476155780=28, srv489656494=79, srv442816343=76, srv749921508=88, srv1165155912=11, srv1342729722=23, srv1563827731=36, srv304260658=69, srv148412557=29, srv1498334414=31, srv605523544=85, srv1646296750=41, srv1930072885=55, srv1695791047=42} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-29 20:57:22,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-29 20:57:22,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-29 20:57:22,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-29 20:57:22,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-29 20:57:22,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-29 20:57:22,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-29 20:57:22,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-29 20:57:22,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-29 20:57:22,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-29 20:57:22,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-29 20:57:22,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-29 20:57:22,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-29 20:57:22,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-29 20:57:22,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-29 20:57:22,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-29 20:57:22,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-29 20:57:22,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-29 20:57:22,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-29 20:57:22,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-29 20:57:22,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-29 20:57:22,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-29 20:57:22,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-29 20:57:22,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-29 20:57:22,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-29 20:57:22,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-29 20:57:22,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-29 20:57:22,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-29 20:57:22,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-29 20:57:22,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-29 20:57:22,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-29 20:57:22,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-29 20:57:22,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-29 20:57:22,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-29 20:57:22,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-29 20:57:22,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-29 20:57:22,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-29 20:57:22,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-29 20:57:22,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-29 20:57:22,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-29 20:57:22,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-29 20:57:22,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-29 20:57:22,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-29 20:57:22,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-29 20:57:22,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-29 20:57:22,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-29 20:57:22,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-29 20:57:22,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-29 20:57:22,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-29 20:57:22,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-29 20:57:22,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-29 20:57:22,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-29 20:57:22,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-29 20:57:22,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-29 20:57:22,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-29 20:57:22,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-29 20:57:22,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-29 20:57:22,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-29 20:57:22,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-29 20:57:22,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-29 20:57:22,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-29 20:57:22,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-29 20:57:22,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-29 20:57:22,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-29 20:57:22,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-29 20:57:22,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-29 20:57:22,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-29 20:57:22,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-29 20:57:22,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-29 20:57:22,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-29 20:57:22,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-29 20:57:22,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-29 20:57:22,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-29 20:57:22,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-29 20:57:22,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-29 20:57:22,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-29 20:57:22,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-29 20:57:22,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-29 20:57:22,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-29 20:57:22,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-29 20:57:22,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-29 20:57:22,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-29 20:57:22,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-29 20:57:22,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-29 20:57:22,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-29 20:57:22,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-29 20:57:22,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-29 20:57:22,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-29 20:57:22,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-29 20:57:22,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-29 20:57:22,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-29 20:57:22,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-29 20:57:22,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-29 20:57:22,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-29 20:57:22,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-29 20:57:22,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-29 20:57:22,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-29 20:57:22,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-29 20:57:22,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-29 20:57:22,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-29 20:57:22,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-29 20:57:22,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-29 20:57:22,178 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0023688968483956715); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-29 20:57:22,178 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-29 20:57:22,178 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=4.2823963699290913E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0023688968483956715); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-29 20:57:22,338 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 236 ms to try 5000 different iterations.  Found a solution that moves 37 regions; Going from a computed imbalance of 4.2823963699290913E-5 to a new imbalance of 8.518843523409391E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.004111111111111111); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-29 20:57:22,338 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv101414353:83 , srv1410001766:83 , srv1851439780:83 , srv1893264417:83 , srv1922445663:83 , srv2092078213:83 , srv260128640:83 , srv605523544:83 , srv1011762074:84 , srv1034548264:84 , srv1052671661:84 , srv1055777773:84 , srv1056537097:84 , srv1087152861:84 , srv1101219882:84 , srv1116838561:84 , srv1131986737:84 , srv1165155912:84 , srv1190571016:84 , srv1202975165:84 , srv1210414209:84 , srv1217506746:84 , srv1227273520:84 , srv1239379441:84 , srv1278571519:84 , srv1279431395:84 , srv1313161803:84 , srv1340182069:84 , srv1342729722:84 , srv1391952820:84 , srv145291149:84 , srv1476155780:84 , srv148412557:84 , srv1498334414:84 , srv1511412242:84 , srv1527121751:84 , srv1530310319:84 , srv1554353059:84 , srv1563827731:84 , srv1574393054:84 , srv1584911188:84 , srv1594394692:84 , srv1597327192:84 , srv1646296750:84 , srv1695791047:84 , srv1708292330:84 , srv1724285571:84 , srv1778830227:84 , srv181542222:84 , srv1885727211:84 , srv1912708385:84 , srv1930072885:84 , srv1944906385:84 , srv2008634590:84 , srv2017276095:84 , srv2046310599:84 , srv2089316631:84 , srv2092165496:84 , srv2092684395:84 , srv2123284948:84 , srv224777915:84 , srv246403091:84 , srv254712458:84 , srv304260658:84 , srv32232895:84 , srv379404136:84 , srv38482267:84 , srv392318122:84 , srv397374002:84 , srv442816343:84 , srv468700632:84 , srv475362811:84 , srv489656494:84 , srv500847637:84 , srv50690895:84 , srv542107106:84 , srv545929022:84 , srv568268577:84 , srv696062847:84 , srv699194962:84 , srv749921508:84 , srv784731111:84 , srv806551161:84 , srv82185574:84 , srv831503806:84 , srv832929462:84 , srv85835611:84 , srv859340292:84 , srv933986155:84 , srv991839427:84 , srv99857140:84 , srv1025746016:85 , srv1254599585:85 , srv1460715039:85 , srv1487126456:85 , srv1760602475:85 , srv177262131:85 , srv395337061:85 , srv807911211:85 , srv192671733:684 }
2024-04-29 20:57:22,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-29 20:57:22,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1034548264=3, srv1116838561=9, srv1912708385=52, srv1460715039=27, srv1052671661=4, srv500847637=80, srv260128640=68, srv859340292=96, srv2046310599=59, srv1055777773=5, srv1391952820=24, srv1487126456=30, srv1584911188=38, srv2008634590=57, srv1087152861=7, srv1011762074=0, srv2123284948=64, srv1893264417=51, srv1202975165=13, srv395337061=74, srv1190571016=12, srv1851439780=49, srv807911211=91, srv2092165496=62, srv1340182069=22, srv1885727211=50, srv38482267=72, srv991839427=98, srv545929022=83, srv1056537097=6, srv1527121751=33, srv1760602475=45, srv2092078213=61, srv1101219882=8, srv699194962=87, srv468700632=77, srv192671733=54, srv1131986737=10, srv696062847=86, srv933986155=97, srv254712458=67, srv1597327192=40, srv1279431395=20, srv1239379441=17, srv1922445663=53, srv246403091=66, srv85835611=95, srv1594394692=39, srv1227273520=16, srv1511412242=32, srv1278571519=19, srv397374002=75, srv806551161=90, srv1217506746=15, srv82185574=92, srv1778830227=47, srv2092684395=63, srv1574393054=37, srv832929462=94, srv99857140=99, srv50690895=81, srv1410001766=25, srv542107106=82, srv1210414209=14, srv181542222=48, srv2017276095=58, srv379404136=71, srv568268577=84, srv1254599585=18, srv1025746016=2, srv1530310319=34, srv1724285571=44, srv1313161803=21, srv32232895=70, srv1944906385=56, srv784731111=89, srv392318122=73, srv177262131=46, srv145291149=26, srv1708292330=43, srv475362811=78, srv2089316631=60, srv831503806=93, srv224777915=65, srv1554353059=35, srv101414353=1, srv1476155780=28, srv489656494=79, srv442816343=76, srv749921508=88, srv1165155912=11, srv1342729722=23, srv1563827731=36, srv304260658=69, srv148412557=29, srv1498334414=31, srv605523544=85, srv1646296750=41, srv1930072885=55, srv1695791047=42} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-29 20:57:22,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-29 20:57:22,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-29 20:57:22,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-29 20:57:22,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-29 20:57:22,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-29 20:57:22,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-29 20:57:22,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-29 20:57:22,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-29 20:57:22,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-29 20:57:22,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-29 20:57:22,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-29 20:57:22,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-29 20:57:22,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-29 20:57:22,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-29 20:57:22,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-29 20:57:22,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-29 20:57:22,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-29 20:57:22,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-29 20:57:22,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-29 20:57:22,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-29 20:57:22,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-29 20:57:22,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-29 20:57:22,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-29 20:57:22,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-29 20:57:22,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-29 20:57:22,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-29 20:57:22,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-29 20:57:22,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-29 20:57:22,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-29 20:57:22,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-29 20:57:22,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-29 20:57:22,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-29 20:57:22,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-29 20:57:22,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-29 20:57:22,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-29 20:57:22,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-29 20:57:22,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-29 20:57:22,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-29 20:57:22,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-29 20:57:22,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-29 20:57:22,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-29 20:57:22,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-29 20:57:22,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-29 20:57:22,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-29 20:57:22,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-29 20:57:22,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-29 20:57:22,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-29 20:57:22,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-29 20:57:22,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-29 20:57:22,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-29 20:57:22,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-29 20:57:22,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-29 20:57:22,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-29 20:57:22,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-29 20:57:22,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-29 20:57:22,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-29 20:57:22,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-29 20:57:22,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-29 20:57:22,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-29 20:57:22,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-29 20:57:22,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-29 20:57:22,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-29 20:57:22,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-29 20:57:22,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-29 20:57:22,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-29 20:57:22,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-29 20:57:22,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-29 20:57:22,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-29 20:57:22,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-29 20:57:22,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-29 20:57:22,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-29 20:57:22,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-29 20:57:22,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-29 20:57:22,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-29 20:57:22,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-29 20:57:22,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-29 20:57:22,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-29 20:57:22,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-29 20:57:22,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-29 20:57:22,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-29 20:57:22,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-29 20:57:22,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-29 20:57:22,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-29 20:57:22,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-29 20:57:22,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-29 20:57:22,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-29 20:57:22,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-29 20:57:22,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-29 20:57:22,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-29 20:57:22,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-29 20:57:22,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-29 20:57:22,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-29 20:57:22,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-29 20:57:22,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-29 20:57:22,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-29 20:57:22,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-29 20:57:22,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-29 20:57:22,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-29 20:57:22,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-29 20:57:22,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-29 20:57:22,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-29 20:57:22,478 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=8.259763985991403E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-29 20:57:22,478 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv101414353:83 , srv1410001766:83 , srv1851439780:83 , srv1893264417:83 , srv1922445663:83 , srv2092078213:83 , srv260128640:83 , srv605523544:83 , srv1011762074:84 , srv1034548264:84 , srv1052671661:84 , srv1055777773:84 , srv1056537097:84 , srv1087152861:84 , srv1101219882:84 , srv1116838561:84 , srv1131986737:84 , srv1165155912:84 , srv1190571016:84 , srv1202975165:84 , srv1210414209:84 , srv1217506746:84 , srv1227273520:84 , srv1239379441:84 , srv1278571519:84 , srv1279431395:84 , srv1313161803:84 , srv1340182069:84 , srv1342729722:84 , srv1391952820:84 , srv145291149:84 , srv1476155780:84 , srv148412557:84 , srv1498334414:84 , srv1511412242:84 , srv1527121751:84 , srv1530310319:84 , srv1554353059:84 , srv1563827731:84 , srv1574393054:84 , srv1584911188:84 , srv1594394692:84 , srv1597327192:84 , srv1646296750:84 , srv1695791047:84 , srv1708292330:84 , srv1724285571:84 , srv1778830227:84 , srv181542222:84 , srv1885727211:84 , srv1912708385:84 , srv1930072885:84 , srv1944906385:84 , srv2008634590:84 , srv2017276095:84 , srv2046310599:84 , srv2089316631:84 , srv2092165496:84 , srv2092684395:84 , srv2123284948:84 , srv224777915:84 , srv246403091:84 , srv254712458:84 , srv304260658:84 , srv32232895:84 , srv379404136:84 , srv38482267:84 , srv392318122:84 , srv397374002:84 , srv442816343:84 , srv468700632:84 , srv475362811:84 , srv489656494:84 , srv500847637:84 , srv50690895:84 , srv542107106:84 , srv545929022:84 , srv568268577:84 , srv696062847:84 , srv699194962:84 , srv749921508:84 , srv784731111:84 , srv806551161:84 , srv82185574:84 , srv831503806:84 , srv832929462:84 , srv85835611:84 , srv859340292:84 , srv933986155:84 , srv991839427:84 , srv99857140:84 , srv1025746016:85 , srv1254599585:85 , srv1460715039:85 , srv1487126456:85 , srv1760602475:85 , srv177262131:85 , srv395337061:85 , srv807911211:85 , srv192671733:684 }
2024-04-29 20:57:22,516 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=289 (was 289), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=273 (was 227) - SystemLoadAverage LEAK? -, ProcessCount=193 (was 193), AvailableMemoryMB=713 (was 1201)
2024-04-29 20:57:22,531 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=289, MaxFileDescriptor=60000, SystemLoadAverage=273, ProcessCount=193, AvailableMemoryMB=712
2024-04-29 20:57:22,531 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-04-29 20:57:22,532 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2024-04-29 20:57:22,541 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1083255344:84 , srv1255140238:84 , srv1364053662:84 , srv136870611:84 , srv1385752637:84 , srv1516218226:84 , srv1535108215:84 , srv1624300495:84 , srv1660401625:84 , srv1662676489:84 , srv1707856749:84 , srv1760215357:84 , srv1822734401:84 , srv2068141603:84 , srv269250740:84 , srv304313495:84 , srv336051376:84 , srv385574825:84 , srv445454446:84 , srv477836895:84 , srv51042038:84 , srv534114536:84 , srv629410113:84 , srv635934710:84 , srv741698884:84 , srv757220644:84 , srv820160339:84 , srv914349510:84 , srv97042489:84 , srv798220480:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2024-04-29 20:57:22,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-29 20:57:22,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1707856749=10, srv1662676489=9, srv1385752637=4, srv1535108215=6, srv1083255344=0, srv136870611=3, srv304313495=15, srv269250740=14, srv1516218226=5, srv635934710=23, srv385574825=17, srv477836895=19, srv1255140238=1, srv1822734401=12, srv798220480=26, srv820160339=27, srv445454446=18, srv97042489=29, srv534114536=21, srv51042038=20, srv629410113=22, srv1660401625=8, srv741698884=24, srv757220644=25, srv914349510=28, srv1364053662=2, srv1760215357=11, srv1624300495=7, srv336051376=16, srv2068141603=13} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-29 20:57:22,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-29 20:57:22,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-29 20:57:22,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-29 20:57:22,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-29 20:57:22,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-29 20:57:22,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-29 20:57:22,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-29 20:57:22,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-29 20:57:22,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-29 20:57:22,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-29 20:57:22,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-29 20:57:22,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-29 20:57:22,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-29 20:57:22,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-29 20:57:22,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-29 20:57:22,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-29 20:57:22,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-29 20:57:22,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-29 20:57:22,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-29 20:57:22,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-29 20:57:22,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-29 20:57:22,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-29 20:57:22,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-29 20:57:22,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-29 20:57:22,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-29 20:57:22,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-29 20:57:22,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-29 20:57:22,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-29 20:57:22,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-29 20:57:22,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-29 20:57:22,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-04-29 20:57:22,577 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1339481291123616, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-29 20:57:22,577 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912615109445304, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1339481291123616, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2024-04-29 20:59:22,565 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 9196667 different iterations.  Found a solution that moves 1852 regions; Going from a computed imbalance of 0.9912615109445304 to a new imbalance of 4.322660390073086E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6859259259259259, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-29 20:59:22,570 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1083255344:90 , srv1255140238:90 , srv1364053662:90 , srv136870611:90 , srv1385752637:90 , srv1516218226:90 , srv1535108215:90 , srv1624300495:90 , srv1660401625:90 , srv1662676489:90 , srv1707856749:90 , srv1760215357:90 , srv1822734401:90 , srv2068141603:90 , srv269250740:90 , srv304313495:90 , srv336051376:90 , srv385574825:90 , srv445454446:90 , srv477836895:90 , srv51042038:90 , srv534114536:90 , srv629410113:90 , srv635934710:90 , srv741698884:90 , srv757220644:90 , srv798220480:90 , srv820160339:90 , srv914349510:90 , srv97042489:90 }
2024-04-29 20:59:22,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-29 20:59:22,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1707856749=10, srv1662676489=9, srv1385752637=4, srv1535108215=6, srv1083255344=0, srv136870611=3, srv304313495=15, srv269250740=14, srv1516218226=5, srv635934710=23, srv385574825=17, srv477836895=19, srv1255140238=1, srv1822734401=12, srv798220480=26, srv820160339=27, srv445454446=18, srv97042489=29, srv534114536=21, srv51042038=20, srv629410113=22, srv1660401625=8, srv741698884=24, srv757220644=25, srv914349510=28, srv1364053662=2, srv1760215357=11, srv1624300495=7, srv336051376=16, srv2068141603=13} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-29 20:59:22,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-29 20:59:22,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-29 20:59:22,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-29 20:59:22,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-29 20:59:22,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-29 20:59:22,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-29 20:59:22,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-29 20:59:22,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-29 20:59:22,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-29 20:59:22,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-29 20:59:22,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-29 20:59:22,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-29 20:59:22,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-29 20:59:22,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-29 20:59:22,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-29 20:59:22,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-29 20:59:22,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-29 20:59:22,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-29 20:59:22,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-29 20:59:22,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-29 20:59:22,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-29 20:59:22,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-29 20:59:22,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-29 20:59:22,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-29 20:59:22,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-29 20:59:22,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-29 20:59:22,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-29 20:59:22,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-29 20:59:22,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-29 20:59:22,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-29 20:59:22,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-04-29 20:59:22,585 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-29 20:59:22,585 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1083255344:90 , srv1255140238:90 , srv1364053662:90 , srv136870611:90 , srv1385752637:90 , srv1516218226:90 , srv1535108215:90 , srv1624300495:90 , srv1660401625:90 , srv1662676489:90 , srv1707856749:90 , srv1760215357:90 , srv1822734401:90 , srv2068141603:90 , srv269250740:90 , srv304313495:90 , srv336051376:90 , srv385574825:90 , srv445454446:90 , srv477836895:90 , srv51042038:90 , srv534114536:90 , srv629410113:90 , srv635934710:90 , srv741698884:90 , srv757220644:90 , srv798220480:90 , srv820160339:90 , srv914349510:90 , srv97042489:90 }
2024-04-29 20:59:22,600 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=289 (was 289), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=174 (was 273), ProcessCount=189 (was 193), AvailableMemoryMB=735 (was 712) - AvailableMemoryMB LEAK? -