2023-05-07 12:53:31,556 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-05-07 12:53:31,901 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-05-07 12:53:32,023 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-05-07 12:53:32,093 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-07 12:53:32,100 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-05-07 12:53:32,113 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=323, ProcessCount=172, AvailableMemoryMB=15404
2023-05-07 12:53:32,115 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-07 12:53:32,116 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-05-07 12:53:32,250 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv102779783:84 , srv1057517088:84 , srv1085355424:84 , srv1111365529:84 , srv1122399765:84 , srv1125194274:84 , srv1143231300:84 , srv1144090033:84 , srv1144332369:84 , srv1153691258:84 , srv1175906908:84 , srv1185829814:84 , srv1186510458:84 , srv123266023:84 , srv1277649640:84 , srv1287099031:84 , srv1289722311:84 , srv1299021200:84 , srv1314521013:84 , srv134421510:84 , srv1367479528:84 , srv1417968603:84 , srv1441861512:84 , srv1443909068:84 , srv1473485050:84 , srv1485343395:84 , srv1488505185:84 , srv1526304477:84 , srv1526607963:84 , srv1578332545:84 , srv1584740396:84 , srv1590643734:84 , srv1592613371:84 , srv1599731962:84 , srv1604412352:84 , srv1605329024:84 , srv1647550163:84 , srv1656105160:84 , srv1659999304:84 , srv1683856816:84 , srv1767917133:84 , srv1771932210:84 , srv1782140314:84 , srv1799155287:84 , srv1810287503:84 , srv1830904623:84 , srv1834740714:84 , srv1843229277:84 , srv1847495322:84 , srv1866156951:84 , srv1883467857:84 , srv1897959500:84 , srv1906975382:84 , srv1914369122:84 , srv1938485091:84 , srv1945985104:84 , srv1955579733:84 , srv1974527974:84 , srv198026991:84 , srv1995322791:84 , srv2022068857:84 , srv2030353196:84 , srv2044741304:84 , srv2064262681:84 , srv2064502001:84 , srv2084134594:84 , srv2107838016:84 , srv258943229:84 , srv260554567:84 , srv273965250:84 , srv301788670:84 , srv303705568:84 , srv379616534:84 , srv46881019:84 , srv474958791:84 , srv485952956:84 , srv537652307:84 , srv574178255:84 , srv578046249:84 , srv602611518:84 , srv604685913:84 , srv60541102:84 , srv630671679:84 , srv631578469:84 , srv659971300:84 , srv677480964:84 , srv705567709:84 , srv706149106:84 , srv709338384:84 , srv712920817:84 , srv759530863:84 , srv772504821:84 , srv812948748:84 , srv817395312:84 , srv831149693:84 , srv851732185:84 , srv937159148:84 , srv943071224:84 , srv953679076:84 , srv1806228723:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-05-07 12:53:32,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-07 12:53:32,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2064262681=64, srv1144332369=8, srv485952956=76, srv260554567=69, srv1782140314=42, srv1289722311=16, srv1834740714=47, srv1955579733=57, srv712920817=90, srv1767917133=40, srv1417968603=21, srv2022068857=61, srv772504821=92, srv812948748=93, srv2044741304=63, srv817395312=94, srv1085355424=2, srv1488505185=26, srv602611518=80, srv1995322791=60, srv2107838016=67, srv631578469=84, srv1526304477=27, srv1057517088=1, srv1647550163=36, srv759530863=91, srv1473485050=24, srv303705568=72, srv943071224=98, srv1153691258=9, srv1367479528=20, srv1125194274=5, srv1656105160=37, srv831149693=95, srv301788670=71, srv1683856816=39, srv1526607963=28, srv2064502001=65, srv1897959500=52, srv60541102=82, srv1584740396=30, srv474958791=75, srv46881019=74, srv1883467857=51, srv1111365529=3, srv1277649640=14, srv1186510458=12, srv123266023=13, srv1592613371=32, srv1847495322=49, srv604685913=81, srv1604412352=34, srv1122399765=4, srv709338384=89, srv1441861512=22, srv1830904623=46, srv1143231300=6, srv1578332545=29, srv1843229277=48, srv677480964=86, srv1938485091=55, srv1299021200=17, srv379616534=73, srv1590643734=31, srv659971300=85, srv134421510=19, srv1443909068=23, srv1605329024=35, srv1175906908=10, srv1906975382=53, srv578046249=79, srv1974527974=58, srv953679076=99, srv1810287503=45, srv1485343395=25, srv198026991=59, srv102779783=0, srv1799155287=43, srv1914369122=54, srv1945985104=56, srv1314521013=18, srv1144090033=7, srv537652307=77, srv258943229=68, srv1185829814=11, srv1771932210=41, srv2030353196=62, srv851732185=96, srv2084134594=66, srv574178255=78, srv705567709=87, srv706149106=88, srv937159148=97, srv1287099031=15, srv1599731962=33, srv1806228723=44, srv1866156951=50, srv1659999304=38, srv273965250=70, srv630671679=83} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-07 12:53:32,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-07 12:53:32,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-07 12:53:32,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-07 12:53:32,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-07 12:53:32,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-07 12:53:32,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-07 12:53:32,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-07 12:53:32,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-07 12:53:32,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-07 12:53:32,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-07 12:53:32,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-07 12:53:32,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-07 12:53:32,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-07 12:53:32,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-07 12:53:32,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-07 12:53:32,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-07 12:53:32,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-07 12:53:32,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-07 12:53:32,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-07 12:53:32,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-07 12:53:32,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-07 12:53:32,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-07 12:53:32,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-07 12:53:32,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-07 12:53:32,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-07 12:53:32,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-07 12:53:32,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-07 12:53:32,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-07 12:53:32,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-07 12:53:32,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-07 12:53:32,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-07 12:53:32,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-07 12:53:32,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-07 12:53:32,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-07 12:53:32,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-07 12:53:32,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-07 12:53:32,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-07 12:53:32,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-07 12:53:32,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-07 12:53:32,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-07 12:53:32,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-07 12:53:32,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-07 12:53:32,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-07 12:53:32,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-07 12:53:32,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-07 12:53:32,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-07 12:53:32,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-07 12:53:32,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-07 12:53:32,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-07 12:53:32,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-07 12:53:32,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-07 12:53:32,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-07 12:53:32,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-07 12:53:32,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-07 12:53:32,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-07 12:53:32,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-07 12:53:32,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-07 12:53:32,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-07 12:53:32,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-07 12:53:32,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-07 12:53:32,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-07 12:53:32,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-07 12:53:32,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-07 12:53:32,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-07 12:53:32,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-07 12:53:32,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-07 12:53:32,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-07 12:53:32,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-07 12:53:32,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-07 12:53:32,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-07 12:53:32,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-07 12:53:32,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-07 12:53:32,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-07 12:53:32,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-07 12:53:32,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-07 12:53:32,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-07 12:53:32,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-07 12:53:32,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-07 12:53:32,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-07 12:53:32,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-07 12:53:32,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-07 12:53:32,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-07 12:53:32,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-07 12:53:32,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-07 12:53:32,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-07 12:53:32,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-07 12:53:32,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-07 12:53:32,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-07 12:53:32,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-07 12:53:32,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-07 12:53:32,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-07 12:53:32,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-07 12:53:32,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-07 12:53:32,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-07 12:53:32,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-07 12:53:32,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-07 12:53:32,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-07 12:53:32,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-07 12:53:32,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-07 12:53:32,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-07 12:53:32,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-07 12:53:32,576 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-07 12:53:32,577 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-07 12:53:32,577 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-07 12:53:33,395 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 997 ms to try 5000 different iterations.  Found a solution that moves 4690 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1810528955255643. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0411073192657841, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.057011232082931844, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5211111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0411073192657841, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17633333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24233333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-07 12:53:33,420 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1175906908:79 , srv1782140314:81 , srv1938485091:81 , srv630671679:81 , srv1153691258:82 , srv1287099031:82 , srv1443909068:82 , srv1914369122:82 , srv1085355424:83 , srv1111365529:83 , srv134421510:83 , srv1592613371:83 , srv1810287503:83 , srv1847495322:83 , srv1866156951:83 , srv1995322791:83 , srv379616534:83 , srv709338384:83 , srv1485343395:84 , srv1526304477:84 , srv1590643734:84 , srv1830904623:84 , srv1906975382:84 , srv1955579733:84 , srv2022068857:84 , srv273965250:84 , srv301788670:84 , srv46881019:84 , srv851732185:84 , srv943071224:84 , srv953679076:84 , srv1144090033:85 , srv123266023:85 , srv1277649640:85 , srv1488505185:85 , srv1605329024:85 , srv1659999304:85 , srv1683856816:85 , srv1771932210:85 , srv2030353196:85 , srv2044741304:85 , srv706149106:85 , srv759530863:85 , srv102779783:86 , srv1057517088:86 , srv1122399765:86 , srv1125194274:86 , srv1441861512:86 , srv1883467857:86 , srv198026991:86 , srv2084134594:86 , srv574178255:86 , srv659971300:86 , srv705567709:86 , srv712920817:86 , srv817395312:86 , srv937159148:86 , srv1185829814:87 , srv1299021200:87 , srv1599731962:87 , srv2064262681:87 , srv2064502001:87 , srv258943229:87 , srv604685913:87 , srv812948748:87 , srv831149693:87 , srv1144332369:88 , srv1417968603:88 , srv1473485050:88 , srv1578332545:88 , srv1656105160:88 , srv1834740714:88 , srv1843229277:88 , srv537652307:88 , srv578046249:88 , srv602611518:88 , srv1143231300:89 , srv1314521013:89 , srv1367479528:89 , srv1767917133:89 , srv2107838016:89 , srv60541102:89 , srv1647550163:90 , srv1799155287:90 , srv1897959500:90 , srv260554567:90 , srv677480964:90 , srv1186510458:91 , srv1526607963:91 , srv1604412352:91 , srv1945985104:91 , srv1974527974:91 , srv485952956:91 , srv474958791:92 , srv631578469:92 , srv772504821:92 , srv1584740396:93 , srv303705568:93 , srv1289722311:94 , srv1806228723:455 }
2023-05-07 12:53:33,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-07 12:53:33,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2064262681=64, srv1144332369=8, srv485952956=76, srv260554567=69, srv1782140314=42, srv1289722311=16, srv1834740714=47, srv1955579733=57, srv712920817=90, srv1767917133=40, srv1417968603=21, srv2022068857=61, srv772504821=92, srv812948748=93, srv2044741304=63, srv817395312=94, srv1085355424=2, srv1488505185=26, srv602611518=80, srv1995322791=60, srv2107838016=67, srv631578469=84, srv1526304477=27, srv1057517088=1, srv1647550163=36, srv759530863=91, srv1473485050=24, srv303705568=72, srv943071224=98, srv1153691258=9, srv1367479528=20, srv1125194274=5, srv1656105160=37, srv831149693=95, srv301788670=71, srv1683856816=39, srv1526607963=28, srv2064502001=65, srv1897959500=52, srv60541102=82, srv1584740396=30, srv474958791=75, srv46881019=74, srv1883467857=51, srv1111365529=3, srv1277649640=14, srv1186510458=12, srv123266023=13, srv1592613371=32, srv1847495322=49, srv604685913=81, srv1604412352=34, srv1122399765=4, srv709338384=89, srv1441861512=22, srv1830904623=46, srv1143231300=6, srv1578332545=29, srv1843229277=48, srv677480964=86, srv1938485091=55, srv1299021200=17, srv379616534=73, srv1590643734=31, srv659971300=85, srv134421510=19, srv1443909068=23, srv1605329024=35, srv1175906908=10, srv1906975382=53, srv578046249=79, srv1974527974=58, srv953679076=99, srv1810287503=45, srv1485343395=25, srv198026991=59, srv102779783=0, srv1799155287=43, srv1914369122=54, srv1945985104=56, srv1314521013=18, srv1144090033=7, srv537652307=77, srv258943229=68, srv1185829814=11, srv1771932210=41, srv2030353196=62, srv851732185=96, srv2084134594=66, srv574178255=78, srv705567709=87, srv706149106=88, srv937159148=97, srv1287099031=15, srv1599731962=33, srv1806228723=44, srv1866156951=50, srv1659999304=38, srv273965250=70, srv630671679=83} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-07 12:53:33,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-07 12:53:33,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-07 12:53:33,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-07 12:53:33,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-07 12:53:33,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-07 12:53:33,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-07 12:53:33,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-07 12:53:33,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-07 12:53:33,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-07 12:53:33,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-07 12:53:33,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-07 12:53:33,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-07 12:53:33,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-07 12:53:33,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-07 12:53:33,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-07 12:53:33,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-07 12:53:33,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-07 12:53:33,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-07 12:53:33,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-07 12:53:33,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-07 12:53:33,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-07 12:53:33,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-07 12:53:33,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-07 12:53:33,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-07 12:53:33,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-07 12:53:33,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-07 12:53:33,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-07 12:53:33,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-07 12:53:33,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-07 12:53:33,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-07 12:53:33,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-07 12:53:33,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-07 12:53:33,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-07 12:53:33,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-07 12:53:33,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-07 12:53:33,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-07 12:53:33,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-07 12:53:33,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-07 12:53:33,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-07 12:53:33,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-07 12:53:33,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-07 12:53:33,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-07 12:53:33,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-07 12:53:33,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-07 12:53:33,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-07 12:53:33,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-07 12:53:33,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-07 12:53:33,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-07 12:53:33,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-07 12:53:33,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-07 12:53:33,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-07 12:53:33,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-07 12:53:33,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-07 12:53:33,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-07 12:53:33,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-07 12:53:33,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-07 12:53:33,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-07 12:53:33,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-07 12:53:33,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-07 12:53:33,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-07 12:53:33,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-07 12:53:33,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-07 12:53:33,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-07 12:53:33,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-07 12:53:33,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-07 12:53:33,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-07 12:53:33,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-07 12:53:33,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-07 12:53:33,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-07 12:53:33,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-07 12:53:33,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-07 12:53:33,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-07 12:53:33,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-07 12:53:33,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-07 12:53:33,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-07 12:53:33,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-07 12:53:33,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-07 12:53:33,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-07 12:53:33,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-07 12:53:33,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-07 12:53:33,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-07 12:53:33,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-07 12:53:33,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-07 12:53:33,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-07 12:53:33,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-07 12:53:33,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-07 12:53:33,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-07 12:53:33,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-07 12:53:33,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-07 12:53:33,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-07 12:53:33,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-07 12:53:33,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-07 12:53:33,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-07 12:53:33,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-07 12:53:33,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-07 12:53:33,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-07 12:53:33,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-07 12:53:33,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-07 12:53:33,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-07 12:53:33,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-07 12:53:33,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-07 12:53:33,634 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0411073192657841, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.057011232082931844, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0411073192657841, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17633333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24233333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-07 12:53:33,634 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-07 12:53:33,634 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18102005544365915, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0411073192657841, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.057011232082931844, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0411073192657841, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17633333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24233333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-07 12:53:33,984 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 494 ms to try 5000 different iterations.  Found a solution that moves 2612 regions; Going from a computed imbalance of 0.18102005544365915 to a new imbalance of 0.013181112632342047. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023421084526534037); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0374465575041463, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2902222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023421084526534037); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010083333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04225, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-07 12:53:34,002 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv303705568:76 , srv1289722311:78 , srv1584740396:78 , srv1526607963:79 , srv474958791:79 , srv485952956:79 , srv677480964:80 , srv1186510458:81 , srv1974527974:81 , srv772504821:81 , srv1144332369:82 , srv1314521013:82 , srv1647550163:82 , srv1767917133:82 , srv1799155287:82 , srv1945985104:82 , srv2107838016:82 , srv258943229:82 , srv260554567:82 , srv537652307:82 , srv60541102:82 , srv1367479528:83 , srv1604412352:83 , srv1843229277:83 , srv1897959500:83 , srv578046249:83 , srv602611518:83 , srv631578469:83 , srv812948748:83 , srv1143231300:84 , srv1599731962:84 , srv1834740714:84 , srv2064262681:84 , srv2064502001:84 , srv937159148:84 , srv1122399765:85 , srv1125194274:85 , srv1277649640:85 , srv1299021200:85 , srv1473485050:85 , srv1771932210:85 , srv2030353196:85 , srv2084134594:85 , srv817395312:85 , srv102779783:86 , srv1057517088:86 , srv1185829814:86 , srv123266023:86 , srv1485343395:86 , srv1488505185:86 , srv1578332545:86 , srv1605329024:86 , srv1656105160:86 , srv1659999304:86 , srv1683856816:86 , srv1883467857:86 , srv659971300:86 , srv706149106:86 , srv1085355424:87 , srv1417968603:87 , srv1441861512:87 , srv1866156951:87 , srv1906975382:87 , srv2022068857:87 , srv2044741304:87 , srv574178255:87 , srv604685913:87 , srv705567709:87 , srv953679076:87 , srv134421510:88 , srv1526304477:88 , srv1592613371:88 , srv1782140314:88 , srv1955579733:88 , srv198026991:88 , srv379616534:88 , srv759530863:88 , srv831149693:88 , srv943071224:88 , srv1153691258:89 , srv1590643734:89 , srv1810287503:89 , srv1914369122:89 , srv46881019:89 , srv851732185:89 , srv1175906908:90 , srv1287099031:90 , srv1443909068:90 , srv1830904623:90 , srv1847495322:90 , srv273965250:90 , srv301788670:90 , srv630671679:90 , srv712920817:90 , srv1111365529:91 , srv1995322791:91 , srv709338384:91 , srv1938485091:93 , srv1144090033:95 , srv1806228723:527 }
2023-05-07 12:53:34,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-07 12:53:34,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2064262681=64, srv1144332369=8, srv485952956=76, srv260554567=69, srv1782140314=42, srv1289722311=16, srv1834740714=47, srv1955579733=57, srv712920817=90, srv1767917133=40, srv1417968603=21, srv2022068857=61, srv772504821=92, srv812948748=93, srv2044741304=63, srv817395312=94, srv1085355424=2, srv1488505185=26, srv602611518=80, srv1995322791=60, srv2107838016=67, srv631578469=84, srv1526304477=27, srv1057517088=1, srv1647550163=36, srv759530863=91, srv1473485050=24, srv303705568=72, srv943071224=98, srv1153691258=9, srv1367479528=20, srv1125194274=5, srv1656105160=37, srv831149693=95, srv301788670=71, srv1683856816=39, srv1526607963=28, srv2064502001=65, srv1897959500=52, srv60541102=82, srv1584740396=30, srv474958791=75, srv46881019=74, srv1883467857=51, srv1111365529=3, srv1277649640=14, srv1186510458=12, srv123266023=13, srv1592613371=32, srv1847495322=49, srv604685913=81, srv1604412352=34, srv1122399765=4, srv709338384=89, srv1441861512=22, srv1830904623=46, srv1143231300=6, srv1578332545=29, srv1843229277=48, srv677480964=86, srv1938485091=55, srv1299021200=17, srv379616534=73, srv1590643734=31, srv659971300=85, srv134421510=19, srv1443909068=23, srv1605329024=35, srv1175906908=10, srv1906975382=53, srv578046249=79, srv1974527974=58, srv953679076=99, srv1810287503=45, srv1485343395=25, srv198026991=59, srv102779783=0, srv1799155287=43, srv1914369122=54, srv1945985104=56, srv1314521013=18, srv1144090033=7, srv537652307=77, srv258943229=68, srv1185829814=11, srv1771932210=41, srv2030353196=62, srv851732185=96, srv2084134594=66, srv574178255=78, srv705567709=87, srv706149106=88, srv937159148=97, srv1287099031=15, srv1599731962=33, srv1806228723=44, srv1866156951=50, srv1659999304=38, srv273965250=70, srv630671679=83} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-07 12:53:34,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-07 12:53:34,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-07 12:53:34,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-07 12:53:34,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-07 12:53:34,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-07 12:53:34,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-07 12:53:34,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-07 12:53:34,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-07 12:53:34,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-07 12:53:34,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-07 12:53:34,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-07 12:53:34,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-07 12:53:34,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-07 12:53:34,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-07 12:53:34,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-07 12:53:34,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-07 12:53:34,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-07 12:53:34,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-07 12:53:34,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-07 12:53:34,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-07 12:53:34,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-07 12:53:34,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-07 12:53:34,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-07 12:53:34,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-07 12:53:34,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-07 12:53:34,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-07 12:53:34,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-07 12:53:34,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-07 12:53:34,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-07 12:53:34,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-07 12:53:34,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-07 12:53:34,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-07 12:53:34,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-07 12:53:34,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-07 12:53:34,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-07 12:53:34,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-07 12:53:34,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-07 12:53:34,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-07 12:53:34,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-07 12:53:34,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-07 12:53:34,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-07 12:53:34,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-07 12:53:34,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-07 12:53:34,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-07 12:53:34,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-07 12:53:34,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-07 12:53:34,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-07 12:53:34,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-07 12:53:34,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-07 12:53:34,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-07 12:53:34,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-07 12:53:34,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-07 12:53:34,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-07 12:53:34,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-07 12:53:34,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-07 12:53:34,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-07 12:53:34,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-07 12:53:34,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-07 12:53:34,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-07 12:53:34,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-07 12:53:34,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-07 12:53:34,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-07 12:53:34,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-07 12:53:34,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-07 12:53:34,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-07 12:53:34,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-07 12:53:34,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-07 12:53:34,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-07 12:53:34,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-07 12:53:34,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-07 12:53:34,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-07 12:53:34,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-07 12:53:34,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-07 12:53:34,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-07 12:53:34,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-07 12:53:34,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-07 12:53:34,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-07 12:53:34,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-07 12:53:34,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-07 12:53:34,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-07 12:53:34,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-07 12:53:34,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-07 12:53:34,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-07 12:53:34,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-07 12:53:34,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-07 12:53:34,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-07 12:53:34,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-07 12:53:34,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-07 12:53:34,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-07 12:53:34,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-07 12:53:34,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-07 12:53:34,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-07 12:53:34,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-07 12:53:34,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-07 12:53:34,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-07 12:53:34,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-07 12:53:34,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-07 12:53:34,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-07 12:53:34,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-07 12:53:34,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-07 12:53:34,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-07 12:53:34,235 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023421084526534037); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0374465575041463, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023421084526534037); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010083333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04225, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-07 12:53:34,235 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-07 12:53:34,235 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.01316282301743027, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023421084526534037); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0374465575041463, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023421084526534037); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010083333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04225, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-07 12:53:34,561 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 496 ms to try 5000 different iterations.  Found a solution that moves 971 regions; Going from a computed imbalance of 0.01316282301743027 to a new imbalance of 6.886414418318056E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00432210984305678); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0068493488921877515); MoveCostFunction : (multiplier=7.0, imbalance=0.1078888888888889, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.00432210984305678); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0036666666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-07 12:53:34,564 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1144090033:77 , srv712920817:81 , srv631578469:82 , srv831149693:82 , srv1111365529:83 , srv1417968603:83 , srv1830904623:83 , srv1938485091:83 , srv301788670:83 , srv604685913:83 , srv1143231300:84 , srv1473485050:84 , srv1578332545:84 , srv1590643734:84 , srv1604412352:84 , srv1656105160:84 , srv1897959500:84 , srv1945985104:84 , srv198026991:84 , srv1995322791:84 , srv46881019:84 , srv705567709:84 , srv709338384:84 , srv759530863:84 , srv102779783:85 , srv1057517088:85 , srv1185829814:85 , srv1186510458:85 , srv1289722311:85 , srv1367479528:85 , srv1526304477:85 , srv1799155287:85 , srv1810287503:85 , srv260554567:85 , srv273965250:85 , srv574178255:85 , srv772504821:85 , srv1287099031:86 , srv1299021200:86 , srv1314521013:86 , srv134421510:86 , srv1441861512:86 , srv1443909068:86 , srv1488505185:86 , srv1584740396:86 , srv1605329024:86 , srv1647550163:86 , srv1834740714:86 , srv1843229277:86 , srv1847495322:86 , srv1883467857:86 , srv2022068857:86 , srv2044741304:86 , srv2084134594:86 , srv474958791:86 , srv578046249:86 , srv602611518:86 , srv60541102:86 , srv659971300:86 , srv851732185:86 , srv943071224:86 , srv953679076:86 , srv1122399765:87 , srv1125194274:87 , srv1144332369:87 , srv1153691258:87 , srv123266023:87 , srv1277649640:87 , srv1592613371:87 , srv1659999304:87 , srv1683856816:87 , srv1767917133:87 , srv1866156951:87 , srv1906975382:87 , srv1914369122:87 , srv1955579733:87 , srv1974527974:87 , srv2030353196:87 , srv2064262681:87 , srv2064502001:87 , srv2107838016:87 , srv379616534:87 , srv485952956:87 , srv630671679:87 , srv677480964:87 , srv706149106:87 , srv937159148:87 , srv1085355424:88 , srv1485343395:88 , srv1526607963:88 , srv1599731962:88 , srv1771932210:88 , srv537652307:88 , srv812948748:88 , srv817395312:88 , srv1175906908:89 , srv1782140314:89 , srv258943229:89 , srv303705568:89 , srv1806228723:514 }
2023-05-07 12:53:34,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-07 12:53:34,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2064262681=64, srv1144332369=8, srv485952956=76, srv260554567=69, srv1782140314=42, srv1289722311=16, srv1834740714=47, srv1955579733=57, srv712920817=90, srv1767917133=40, srv1417968603=21, srv2022068857=61, srv772504821=92, srv812948748=93, srv2044741304=63, srv817395312=94, srv1085355424=2, srv1488505185=26, srv602611518=80, srv1995322791=60, srv2107838016=67, srv631578469=84, srv1526304477=27, srv1057517088=1, srv1647550163=36, srv759530863=91, srv1473485050=24, srv303705568=72, srv943071224=98, srv1153691258=9, srv1367479528=20, srv1125194274=5, srv1656105160=37, srv831149693=95, srv301788670=71, srv1683856816=39, srv1526607963=28, srv2064502001=65, srv1897959500=52, srv60541102=82, srv1584740396=30, srv474958791=75, srv46881019=74, srv1883467857=51, srv1111365529=3, srv1277649640=14, srv1186510458=12, srv123266023=13, srv1592613371=32, srv1847495322=49, srv604685913=81, srv1604412352=34, srv1122399765=4, srv709338384=89, srv1441861512=22, srv1830904623=46, srv1143231300=6, srv1578332545=29, srv1843229277=48, srv677480964=86, srv1938485091=55, srv1299021200=17, srv379616534=73, srv1590643734=31, srv659971300=85, srv134421510=19, srv1443909068=23, srv1605329024=35, srv1175906908=10, srv1906975382=53, srv578046249=79, srv1974527974=58, srv953679076=99, srv1810287503=45, srv1485343395=25, srv198026991=59, srv102779783=0, srv1799155287=43, srv1914369122=54, srv1945985104=56, srv1314521013=18, srv1144090033=7, srv537652307=77, srv258943229=68, srv1185829814=11, srv1771932210=41, srv2030353196=62, srv851732185=96, srv2084134594=66, srv574178255=78, srv705567709=87, srv706149106=88, srv937159148=97, srv1287099031=15, srv1599731962=33, srv1806228723=44, srv1866156951=50, srv1659999304=38, srv273965250=70, srv630671679=83} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-07 12:53:34,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-07 12:53:34,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-07 12:53:34,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-07 12:53:34,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-07 12:53:34,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-07 12:53:34,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-07 12:53:34,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-07 12:53:34,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-07 12:53:34,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-07 12:53:34,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-07 12:53:34,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-07 12:53:34,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-07 12:53:34,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-07 12:53:34,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-07 12:53:34,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-07 12:53:34,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-07 12:53:34,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-07 12:53:34,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-07 12:53:34,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-07 12:53:34,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-07 12:53:34,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-07 12:53:34,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-07 12:53:34,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-07 12:53:34,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-07 12:53:34,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-07 12:53:34,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-07 12:53:34,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-07 12:53:34,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-07 12:53:34,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-07 12:53:34,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-07 12:53:34,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-07 12:53:34,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-07 12:53:34,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-07 12:53:34,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-07 12:53:34,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-07 12:53:34,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-07 12:53:34,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-07 12:53:34,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-07 12:53:34,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-07 12:53:34,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-07 12:53:34,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-07 12:53:34,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-07 12:53:34,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-07 12:53:34,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-07 12:53:34,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-07 12:53:34,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-07 12:53:34,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-07 12:53:34,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-07 12:53:34,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-07 12:53:34,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-07 12:53:34,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-07 12:53:34,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-07 12:53:34,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-07 12:53:34,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-07 12:53:34,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-07 12:53:34,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-07 12:53:34,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-07 12:53:34,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-07 12:53:34,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-07 12:53:34,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-07 12:53:34,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-07 12:53:34,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-07 12:53:34,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-07 12:53:34,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-07 12:53:34,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-07 12:53:34,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-07 12:53:34,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-07 12:53:34,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-07 12:53:34,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-07 12:53:34,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-07 12:53:34,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-07 12:53:34,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-07 12:53:34,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-07 12:53:34,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-07 12:53:34,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-07 12:53:34,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-07 12:53:34,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-07 12:53:34,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-07 12:53:34,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-07 12:53:34,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-07 12:53:34,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-07 12:53:34,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-07 12:53:34,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-07 12:53:34,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-07 12:53:34,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-07 12:53:34,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-07 12:53:34,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-07 12:53:34,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-07 12:53:34,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-07 12:53:34,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-07 12:53:34,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-07 12:53:34,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-07 12:53:34,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-07 12:53:34,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-07 12:53:34,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-07 12:53:34,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-07 12:53:34,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-07 12:53:34,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-07 12:53:34,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-07 12:53:34,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-07 12:53:34,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-07 12:53:34,762 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00432210984305678); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0068493488921877515); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.00432210984305678); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0036666666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-07 12:53:34,763 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-07 12:53:34,763 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=6.818423545119984E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00432210984305678); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0068493488921877515); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.00432210984305678); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0036666666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-07 12:53:35,077 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 450 ms to try 5000 different iterations.  Found a solution that moves 223 regions; Going from a computed imbalance of 6.818423545119984E-4 to a new imbalance of 1.2063958173565962E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.23782800878924E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017727200782610348); MoveCostFunction : (multiplier=7.0, imbalance=0.024777777777777777); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.23782800878924E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-07 12:53:35,079 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1441861512:82 , srv1599731962:83 , srv1767917133:83 , srv1782140314:83 , srv1847495322:83 , srv1995322791:83 , srv273965250:83 , srv379616534:83 , srv817395312:83 , srv851732185:83 , srv102779783:84 , srv1085355424:84 , srv1122399765:84 , srv1153691258:84 , srv1175906908:84 , srv1185829814:84 , srv123266023:84 , srv1287099031:84 , srv1289722311:84 , srv1299021200:84 , srv1417968603:84 , srv1443909068:84 , srv1485343395:84 , srv1526607963:84 , srv1578332545:84 , srv1590643734:84 , srv1592613371:84 , srv1647550163:84 , srv1656105160:84 , srv1659999304:84 , srv1683856816:84 , srv1771932210:84 , srv1843229277:84 , srv1883467857:84 , srv1906975382:84 , srv1914369122:84 , srv1955579733:84 , srv1974527974:84 , srv198026991:84 , srv2022068857:84 , srv2044741304:84 , srv2064262681:84 , srv2064502001:84 , srv2107838016:84 , srv258943229:84 , srv303705568:84 , srv537652307:84 , srv630671679:84 , srv706149106:84 , srv709338384:84 , srv712920817:84 , srv812948748:84 , srv943071224:84 , srv1057517088:85 , srv1111365529:85 , srv1125194274:85 , srv1143231300:85 , srv1144090033:85 , srv1144332369:85 , srv1186510458:85 , srv1277649640:85 , srv1314521013:85 , srv134421510:85 , srv1367479528:85 , srv1473485050:85 , srv1488505185:85 , srv1526304477:85 , srv1584740396:85 , srv1604412352:85 , srv1605329024:85 , srv1810287503:85 , srv1834740714:85 , srv1866156951:85 , srv1897959500:85 , srv1938485091:85 , srv1945985104:85 , srv2030353196:85 , srv2084134594:85 , srv260554567:85 , srv301788670:85 , srv46881019:85 , srv474958791:85 , srv485952956:85 , srv574178255:85 , srv578046249:85 , srv602611518:85 , srv604685913:85 , srv60541102:85 , srv631578469:85 , srv659971300:85 , srv677480964:85 , srv759530863:85 , srv772504821:85 , srv831149693:85 , srv937159148:85 , srv953679076:85 , srv1799155287:86 , srv1830904623:86 , srv705567709:86 , srv1806228723:646 }
2023-05-07 12:53:35,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-07 12:53:35,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2064262681=64, srv1144332369=8, srv485952956=76, srv260554567=69, srv1782140314=42, srv1289722311=16, srv1834740714=47, srv1955579733=57, srv712920817=90, srv1767917133=40, srv1417968603=21, srv2022068857=61, srv772504821=92, srv812948748=93, srv2044741304=63, srv817395312=94, srv1085355424=2, srv1488505185=26, srv602611518=80, srv1995322791=60, srv2107838016=67, srv631578469=84, srv1526304477=27, srv1057517088=1, srv1647550163=36, srv759530863=91, srv1473485050=24, srv303705568=72, srv943071224=98, srv1153691258=9, srv1367479528=20, srv1125194274=5, srv1656105160=37, srv831149693=95, srv301788670=71, srv1683856816=39, srv1526607963=28, srv2064502001=65, srv1897959500=52, srv60541102=82, srv1584740396=30, srv474958791=75, srv46881019=74, srv1883467857=51, srv1111365529=3, srv1277649640=14, srv1186510458=12, srv123266023=13, srv1592613371=32, srv1847495322=49, srv604685913=81, srv1604412352=34, srv1122399765=4, srv709338384=89, srv1441861512=22, srv1830904623=46, srv1143231300=6, srv1578332545=29, srv1843229277=48, srv677480964=86, srv1938485091=55, srv1299021200=17, srv379616534=73, srv1590643734=31, srv659971300=85, srv134421510=19, srv1443909068=23, srv1605329024=35, srv1175906908=10, srv1906975382=53, srv578046249=79, srv1974527974=58, srv953679076=99, srv1810287503=45, srv1485343395=25, srv198026991=59, srv102779783=0, srv1799155287=43, srv1914369122=54, srv1945985104=56, srv1314521013=18, srv1144090033=7, srv537652307=77, srv258943229=68, srv1185829814=11, srv1771932210=41, srv2030353196=62, srv851732185=96, srv2084134594=66, srv574178255=78, srv705567709=87, srv706149106=88, srv937159148=97, srv1287099031=15, srv1599731962=33, srv1806228723=44, srv1866156951=50, srv1659999304=38, srv273965250=70, srv630671679=83} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-07 12:53:35,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-07 12:53:35,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-07 12:53:35,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-07 12:53:35,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-07 12:53:35,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-07 12:53:35,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-07 12:53:35,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-07 12:53:35,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-07 12:53:35,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-07 12:53:35,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-07 12:53:35,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-07 12:53:35,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-07 12:53:35,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-07 12:53:35,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-07 12:53:35,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-07 12:53:35,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-07 12:53:35,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-07 12:53:35,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-07 12:53:35,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-07 12:53:35,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-07 12:53:35,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-07 12:53:35,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-07 12:53:35,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-07 12:53:35,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-07 12:53:35,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-07 12:53:35,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-07 12:53:35,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-07 12:53:35,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-07 12:53:35,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-07 12:53:35,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-07 12:53:35,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-07 12:53:35,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-07 12:53:35,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-07 12:53:35,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-07 12:53:35,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-07 12:53:35,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-07 12:53:35,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-07 12:53:35,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-07 12:53:35,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-07 12:53:35,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-07 12:53:35,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-07 12:53:35,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-07 12:53:35,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-07 12:53:35,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-07 12:53:35,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-07 12:53:35,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-07 12:53:35,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-07 12:53:35,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-07 12:53:35,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-07 12:53:35,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-07 12:53:35,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-07 12:53:35,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-07 12:53:35,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-07 12:53:35,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-07 12:53:35,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-07 12:53:35,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-07 12:53:35,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-07 12:53:35,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-07 12:53:35,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-07 12:53:35,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-07 12:53:35,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-07 12:53:35,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-07 12:53:35,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-07 12:53:35,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-07 12:53:35,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-07 12:53:35,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-07 12:53:35,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-07 12:53:35,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-07 12:53:35,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-07 12:53:35,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-07 12:53:35,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-07 12:53:35,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-07 12:53:35,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-07 12:53:35,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-07 12:53:35,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-07 12:53:35,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-07 12:53:35,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-07 12:53:35,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-07 12:53:35,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-07 12:53:35,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-07 12:53:35,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-07 12:53:35,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-07 12:53:35,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-07 12:53:35,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-07 12:53:35,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-07 12:53:35,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-07 12:53:35,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-07 12:53:35,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-07 12:53:35,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-07 12:53:35,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-07 12:53:35,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-07 12:53:35,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-07 12:53:35,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-07 12:53:35,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-07 12:53:35,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-07 12:53:35,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-07 12:53:35,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-07 12:53:35,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-07 12:53:35,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-07 12:53:35,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-07 12:53:35,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-07 12:53:35,253 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.050247879939809E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.23782800878924E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017727200782610348); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.23782800878924E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-07 12:53:35,253 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1441861512:82 , srv1599731962:83 , srv1767917133:83 , srv1782140314:83 , srv1847495322:83 , srv1995322791:83 , srv273965250:83 , srv379616534:83 , srv817395312:83 , srv851732185:83 , srv102779783:84 , srv1085355424:84 , srv1122399765:84 , srv1153691258:84 , srv1175906908:84 , srv1185829814:84 , srv123266023:84 , srv1287099031:84 , srv1289722311:84 , srv1299021200:84 , srv1417968603:84 , srv1443909068:84 , srv1485343395:84 , srv1526607963:84 , srv1578332545:84 , srv1590643734:84 , srv1592613371:84 , srv1647550163:84 , srv1656105160:84 , srv1659999304:84 , srv1683856816:84 , srv1771932210:84 , srv1843229277:84 , srv1883467857:84 , srv1906975382:84 , srv1914369122:84 , srv1955579733:84 , srv1974527974:84 , srv198026991:84 , srv2022068857:84 , srv2044741304:84 , srv2064262681:84 , srv2064502001:84 , srv2107838016:84 , srv258943229:84 , srv303705568:84 , srv537652307:84 , srv630671679:84 , srv706149106:84 , srv709338384:84 , srv712920817:84 , srv812948748:84 , srv943071224:84 , srv1057517088:85 , srv1111365529:85 , srv1125194274:85 , srv1143231300:85 , srv1144090033:85 , srv1144332369:85 , srv1186510458:85 , srv1277649640:85 , srv1314521013:85 , srv134421510:85 , srv1367479528:85 , srv1473485050:85 , srv1488505185:85 , srv1526304477:85 , srv1584740396:85 , srv1604412352:85 , srv1605329024:85 , srv1810287503:85 , srv1834740714:85 , srv1866156951:85 , srv1897959500:85 , srv1938485091:85 , srv1945985104:85 , srv2030353196:85 , srv2084134594:85 , srv260554567:85 , srv301788670:85 , srv46881019:85 , srv474958791:85 , srv485952956:85 , srv574178255:85 , srv578046249:85 , srv602611518:85 , srv604685913:85 , srv60541102:85 , srv631578469:85 , srv659971300:85 , srv677480964:85 , srv759530863:85 , srv772504821:85 , srv831149693:85 , srv937159148:85 , srv953679076:85 , srv1799155287:86 , srv1830904623:86 , srv705567709:86 , srv1806228723:646 }
2023-05-07 12:53:35,288 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=323 (was 323), ProcessCount=172 (was 172), AvailableMemoryMB=14532 (was 15404)
2023-05-07 12:53:35,303 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=262, MaxFileDescriptor=60000, SystemLoadAverage=323, ProcessCount=172, AvailableMemoryMB=14530
2023-05-07 12:53:35,304 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-07 12:53:35,305 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-05-07 12:53:35,315 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1015649999:84 , srv106808886:84 , srv1243797997:84 , srv1445118109:84 , srv14554328:84 , srv1656650134:84 , srv171527478:84 , srv1787334148:84 , srv1832527743:84 , srv1868471661:84 , srv1874136641:84 , srv1953056944:84 , srv1993707427:84 , srv206616260:84 , srv2085605697:84 , srv212275834:84 , srv240659616:84 , srv27598184:84 , srv416630544:84 , srv424496313:84 , srv453310113:84 , srv525343905:84 , srv580115255:84 , srv580270552:84 , srv75657362:84 , srv787884739:84 , srv813194253:84 , srv818442426:84 , srv849480074:84 , srv770467700:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-05-07 12:53:35,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-07 12:53:35,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv813194253=27, srv206616260=13, srv1874136641=10, srv580115255=22, srv453310113=20, srv27598184=17, srv818442426=28, srv1953056944=11, srv1445118109=3, srv770467700=25, srv849480074=29, srv171527478=6, srv1868471661=9, srv75657362=24, srv1015649999=0, srv416630544=18, srv424496313=19, srv787884739=26, srv212275834=15, srv525343905=21, srv240659616=16, srv1656650134=5, srv14554328=4, srv1993707427=12, srv1787334148=7, srv2085605697=14, srv106808886=1, srv1243797997=2, srv580270552=23, srv1832527743=8} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-07 12:53:35,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-07 12:53:35,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-07 12:53:35,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-07 12:53:35,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-07 12:53:35,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-07 12:53:35,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-07 12:53:35,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-07 12:53:35,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-07 12:53:35,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-07 12:53:35,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-07 12:53:35,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-07 12:53:35,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-07 12:53:35,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-07 12:53:35,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-07 12:53:35,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-07 12:53:35,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-07 12:53:35,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-07 12:53:35,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-07 12:53:35,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-07 12:53:35,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-07 12:53:35,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-07 12:53:35,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-07 12:53:35,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-07 12:53:35,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-07 12:53:35,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-07 12:53:35,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-07 12:53:35,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-07 12:53:35,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-07 12:53:35,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-07 12:53:35,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-07 12:53:35,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-05-07 12:53:35,377 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0626733518432585, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-07 12:53:35,377 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912390524949466, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0626733518432585, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-05-07 12:55:35,337 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 7048114 different iterations.  Found a solution that moves 1850 regions; Going from a computed imbalance of 0.9912390524949466 to a new imbalance of 4.317992290299789E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6851851851851852, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-07 12:55:35,344 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1015649999:90 , srv106808886:90 , srv1243797997:90 , srv1445118109:90 , srv14554328:90 , srv1656650134:90 , srv171527478:90 , srv1787334148:90 , srv1832527743:90 , srv1868471661:90 , srv1874136641:90 , srv1953056944:90 , srv1993707427:90 , srv206616260:90 , srv2085605697:90 , srv212275834:90 , srv240659616:90 , srv27598184:90 , srv416630544:90 , srv424496313:90 , srv453310113:90 , srv525343905:90 , srv580115255:90 , srv580270552:90 , srv75657362:90 , srv770467700:90 , srv787884739:90 , srv813194253:90 , srv818442426:90 , srv849480074:90 }
2023-05-07 12:55:35,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-07 12:55:35,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv813194253=27, srv206616260=13, srv1874136641=10, srv580115255=22, srv453310113=20, srv27598184=17, srv818442426=28, srv1953056944=11, srv1445118109=3, srv770467700=25, srv849480074=29, srv171527478=6, srv1868471661=9, srv75657362=24, srv1015649999=0, srv416630544=18, srv424496313=19, srv787884739=26, srv212275834=15, srv525343905=21, srv240659616=16, srv1656650134=5, srv14554328=4, srv1993707427=12, srv1787334148=7, srv2085605697=14, srv106808886=1, srv1243797997=2, srv580270552=23, srv1832527743=8} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-07 12:55:35,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-07 12:55:35,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-07 12:55:35,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-07 12:55:35,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-07 12:55:35,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-07 12:55:35,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-07 12:55:35,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-07 12:55:35,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-07 12:55:35,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-07 12:55:35,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-07 12:55:35,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-07 12:55:35,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-07 12:55:35,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-07 12:55:35,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-07 12:55:35,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-07 12:55:35,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-07 12:55:35,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-07 12:55:35,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-07 12:55:35,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-07 12:55:35,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-07 12:55:35,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-07 12:55:35,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-07 12:55:35,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-07 12:55:35,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-07 12:55:35,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-07 12:55:35,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-07 12:55:35,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-07 12:55:35,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-07 12:55:35,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-07 12:55:35,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-07 12:55:35,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-05-07 12:55:35,362 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-07 12:55:35,363 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1015649999:90 , srv106808886:90 , srv1243797997:90 , srv1445118109:90 , srv14554328:90 , srv1656650134:90 , srv171527478:90 , srv1787334148:90 , srv1832527743:90 , srv1868471661:90 , srv1874136641:90 , srv1953056944:90 , srv1993707427:90 , srv206616260:90 , srv2085605697:90 , srv212275834:90 , srv240659616:90 , srv27598184:90 , srv416630544:90 , srv424496313:90 , srv453310113:90 , srv525343905:90 , srv580115255:90 , srv580270552:90 , srv75657362:90 , srv770467700:90 , srv787884739:90 , srv813194253:90 , srv818442426:90 , srv849480074:90 }
2023-05-07 12:55:35,378 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 262), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=197 (was 323), ProcessCount=169 (was 172), AvailableMemoryMB=14829 (was 14530) - AvailableMemoryMB LEAK? -