2024-02-08 17:54:18,710 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2024-02-08 17:54:18,980 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2024-02-08 17:54:19,078 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2024-02-08 17:54:19,134 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-02-08 17:54:19,142 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-02-08 17:54:19,155 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=269, ProcessCount=175, AvailableMemoryMB=2219
2024-02-08 17:54:19,157 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-02-08 17:54:19,158 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-02-08 17:54:19,285 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1029743763:84 , srv1033987955:84 , srv1052697473:84 , srv1132191835:84 , srv1186022527:84 , srv1198440602:84 , srv1205122848:84 , srv1221758274:84 , srv1222392278:84 , srv1229105198:84 , srv1236295989:84 , srv123897810:84 , srv125235427:84 , srv1259585224:84 , srv1268392819:84 , srv1302117112:84 , srv1311672063:84 , srv1322685828:84 , srv1333246047:84 , srv1341192625:84 , srv1351224093:84 , srv1358569171:84 , srv1363386592:84 , srv1375833022:84 , srv1380037609:84 , srv139475738:84 , srv1399167940:84 , srv1403066380:84 , srv1403299137:84 , srv1408834784:84 , srv1437178033:84 , srv1445813830:84 , srv1453949934:84 , srv146325980:84 , srv1464045783:84 , srv1479091340:84 , srv1491776671:84 , srv1534175113:84 , srv1554473777:84 , srv1566385257:84 , srv1594463773:84 , srv1603400360:84 , srv1615751628:84 , srv1616613396:84 , srv1643705456:84 , srv1650856945:84 , srv1672132438:84 , srv168447645:84 , srv1687847405:84 , srv1765082499:84 , srv1767099283:84 , srv181739199:84 , srv182771726:84 , srv1830584331:84 , srv1836653503:84 , srv1839898159:84 , srv1858397318:84 , srv1870596302:84 , srv1875195679:84 , srv1914294513:84 , srv1936504850:84 , srv1937498879:84 , srv194039828:84 , srv1942348226:84 , srv1972806299:84 , srv1992580120:84 , srv2005748246:84 , srv2018602364:84 , srv2043081173:84 , srv2060552614:84 , srv2075286304:84 , srv2089081200:84 , srv241202465:84 , srv241838298:84 , srv25395936:84 , srv259423077:84 , srv3248236:84 , srv3371011:84 , srv364374670:84 , srv408911953:84 , srv417264694:84 , srv441114406:84 , srv451015856:84 , srv4530785:84 , srv538966467:84 , srv625314694:84 , srv641731055:84 , srv667224089:84 , srv679782640:84 , srv681562098:84 , srv745375997:84 , srv772633831:84 , srv811415096:84 , srv868730494:84 , srv877831921:84 , srv900965766:84 , srv919805028:84 , srv921885413:84 , srv945981964:84 , srv1442351400:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2024-02-08 17:54:19,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-08 17:54:19,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1268392819=14, srv1616613396=44, srv919805028=97, srv1442351400=31, srv1615751628=43, srv408911953=80, srv1445813830=32, srv364374670=79, srv1534175113=38, srv139475738=25, srv2060552614=70, srv1375833022=23, srv441114406=82, srv1222392278=8, srv1566385257=40, srv811415096=93, srv877831921=95, srv1033987955=1, srv1351224093=20, srv1236295989=10, srv241202465=73, srv2018602364=68, srv1830584331=54, srv1221758274=7, srv1870596302=58, srv679782640=89, srv194039828=63, srv868730494=94, srv2005748246=67, srv146325980=34, srv1132191835=3, srv3248236=77, srv1937498879=62, srv1437178033=30, srv1836653503=55, srv168447645=48, srv1358569171=21, srv4530785=84, srv1643705456=45, srv538966467=85, srv259423077=76, srv1491776671=37, srv1875195679=59, srv1914294513=60, srv125235427=12, srv1839898159=56, srv451015856=83, srv1341192625=19, srv417264694=81, srv1479091340=36, srv2075286304=71, srv1858397318=57, srv1186022527=4, srv25395936=75, srv1650856945=46, srv2043081173=69, srv1198440602=5, srv1029743763=0, srv182771726=53, srv681562098=90, srv1453949934=33, srv1942348226=64, srv1972806299=65, srv1403299137=28, srv1765082499=50, srv1687847405=49, srv921885413=98, srv1333246047=18, srv1380037609=24, srv1205122848=6, srv1311672063=16, srv1594463773=41, srv2089081200=72, srv3371011=78, srv1992580120=66, srv1464045783=35, srv745375997=91, srv181739199=52, srv641731055=87, srv241838298=74, srv1052697473=2, srv667224089=88, srv900965766=96, srv1399167940=26, srv1554473777=39, srv1302117112=15, srv1322685828=17, srv1229105198=9, srv123897810=11, srv772633831=92, srv1259585224=13, srv1403066380=27, srv1603400360=42, srv1363386592=22, srv625314694=86, srv1767099283=51, srv1672132438=47, srv1936504850=61, srv945981964=99, srv1408834784=29} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-08 17:54:19,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-08 17:54:19,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-08 17:54:19,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-08 17:54:19,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-08 17:54:19,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-08 17:54:19,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-08 17:54:19,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-08 17:54:19,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-08 17:54:19,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-08 17:54:19,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-08 17:54:19,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-08 17:54:19,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-08 17:54:19,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-08 17:54:19,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-08 17:54:19,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-08 17:54:19,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-08 17:54:19,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-08 17:54:19,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-08 17:54:19,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-08 17:54:19,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-08 17:54:19,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-08 17:54:19,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-08 17:54:19,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-08 17:54:19,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-08 17:54:19,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-08 17:54:19,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-08 17:54:19,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-08 17:54:19,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-08 17:54:19,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-08 17:54:19,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-08 17:54:19,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-02-08 17:54:19,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-02-08 17:54:19,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-02-08 17:54:19,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-02-08 17:54:19,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-02-08 17:54:19,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-02-08 17:54:19,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-02-08 17:54:19,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-02-08 17:54:19,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-02-08 17:54:19,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-02-08 17:54:19,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-02-08 17:54:19,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-02-08 17:54:19,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-02-08 17:54:19,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-02-08 17:54:19,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-02-08 17:54:19,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-02-08 17:54:19,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-02-08 17:54:19,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-02-08 17:54:19,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-02-08 17:54:19,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-02-08 17:54:19,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-02-08 17:54:19,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-02-08 17:54:19,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-02-08 17:54:19,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-02-08 17:54:19,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-02-08 17:54:19,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-02-08 17:54:19,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-02-08 17:54:19,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-02-08 17:54:19,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-02-08 17:54:19,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-02-08 17:54:19,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-02-08 17:54:19,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-02-08 17:54:19,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-02-08 17:54:19,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-02-08 17:54:19,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-02-08 17:54:19,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-02-08 17:54:19,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-02-08 17:54:19,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-02-08 17:54:19,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-02-08 17:54:19,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-02-08 17:54:19,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-02-08 17:54:19,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-02-08 17:54:19,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-02-08 17:54:19,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-02-08 17:54:19,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-02-08 17:54:19,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-02-08 17:54:19,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-02-08 17:54:19,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-02-08 17:54:19,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-02-08 17:54:19,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-02-08 17:54:19,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-02-08 17:54:19,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-02-08 17:54:19,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-02-08 17:54:19,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-02-08 17:54:19,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-02-08 17:54:19,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-02-08 17:54:19,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-02-08 17:54:19,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-02-08 17:54:19,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-02-08 17:54:19,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-02-08 17:54:19,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-02-08 17:54:19,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-02-08 17:54:19,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-02-08 17:54:19,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-02-08 17:54:19,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-02-08 17:54:19,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-02-08 17:54:19,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-02-08 17:54:19,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-02-08 17:54:19,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-02-08 17:54:19,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-02-08 17:54:19,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-02-08 17:54:19,553 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-08 17:54:19,554 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-02-08 17:54:19,554 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-02-08 17:54:20,326 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 905 ms to try 5000 different iterations.  Found a solution that moves 4686 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1816074911055612. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.036764994167131866, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05320270197268334, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5206666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.036764994167131866, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17716666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24058333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-08 17:54:20,342 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1198440602:83 , srv168447645:83 , srv945981964:83 , srv1033987955:84 , srv1222392278:84 , srv1479091340:84 , srv1687847405:84 , srv1765082499:84 , srv182771726:84 , srv1830584331:84 , srv194039828:84 , srv123897810:85 , srv1491776671:85 , srv1566385257:85 , srv1594463773:85 , srv1615751628:85 , srv1875195679:85 , srv2005748246:85 , srv2018602364:85 , srv2060552614:85 , srv241202465:85 , srv241838298:85 , srv259423077:85 , srv3371011:85 , srv900965766:85 , srv1052697473:86 , srv1259585224:86 , srv1302117112:86 , srv1351224093:86 , srv1363386592:86 , srv139475738:86 , srv1399167940:86 , srv1403299137:86 , srv1437178033:86 , srv1554473777:86 , srv1643705456:86 , srv1870596302:86 , srv1936504850:86 , srv1972806299:86 , srv1992580120:86 , srv2043081173:86 , srv2075286304:86 , srv408911953:86 , srv441114406:86 , srv451015856:86 , srv4530785:86 , srv625314694:86 , srv667224089:86 , srv679782640:86 , srv745375997:86 , srv868730494:86 , srv877831921:86 , srv1186022527:87 , srv1205122848:87 , srv1229105198:87 , srv1322685828:87 , srv1375833022:87 , srv1380037609:87 , srv1445813830:87 , srv146325980:87 , srv1603400360:87 , srv1616613396:87 , srv1767099283:87 , srv181739199:87 , srv1839898159:87 , srv3248236:87 , srv364374670:87 , srv811415096:87 , srv919805028:87 , srv921885413:87 , srv1029743763:88 , srv1236295989:88 , srv125235427:88 , srv1333246047:88 , srv1358569171:88 , srv1408834784:88 , srv1453949934:88 , srv1464045783:88 , srv1650856945:88 , srv1937498879:88 , srv1942348226:88 , srv417264694:88 , srv641731055:88 , srv1268392819:89 , srv1311672063:89 , srv1403066380:89 , srv1534175113:89 , srv1836653503:89 , srv1858397318:89 , srv538966467:89 , srv681562098:89 , srv1132191835:90 , srv1221758274:90 , srv1672132438:90 , srv2089081200:90 , srv1341192625:91 , srv772633831:91 , srv25395936:92 , srv1914294513:94 , srv1442351400:417 }
2024-02-08 17:54:20,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-08 17:54:20,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1268392819=14, srv1616613396=44, srv919805028=97, srv1442351400=31, srv1615751628=43, srv408911953=80, srv1445813830=32, srv364374670=79, srv1534175113=38, srv139475738=25, srv2060552614=70, srv1375833022=23, srv441114406=82, srv1222392278=8, srv1566385257=40, srv811415096=93, srv877831921=95, srv1033987955=1, srv1351224093=20, srv1236295989=10, srv241202465=73, srv2018602364=68, srv1830584331=54, srv1221758274=7, srv1870596302=58, srv679782640=89, srv194039828=63, srv868730494=94, srv2005748246=67, srv146325980=34, srv1132191835=3, srv3248236=77, srv1937498879=62, srv1437178033=30, srv1836653503=55, srv168447645=48, srv1358569171=21, srv4530785=84, srv1643705456=45, srv538966467=85, srv259423077=76, srv1491776671=37, srv1875195679=59, srv1914294513=60, srv125235427=12, srv1839898159=56, srv451015856=83, srv1341192625=19, srv417264694=81, srv1479091340=36, srv2075286304=71, srv1858397318=57, srv1186022527=4, srv25395936=75, srv1650856945=46, srv2043081173=69, srv1198440602=5, srv1029743763=0, srv182771726=53, srv681562098=90, srv1453949934=33, srv1942348226=64, srv1972806299=65, srv1403299137=28, srv1765082499=50, srv1687847405=49, srv921885413=98, srv1333246047=18, srv1380037609=24, srv1205122848=6, srv1311672063=16, srv1594463773=41, srv2089081200=72, srv3371011=78, srv1992580120=66, srv1464045783=35, srv745375997=91, srv181739199=52, srv641731055=87, srv241838298=74, srv1052697473=2, srv667224089=88, srv900965766=96, srv1399167940=26, srv1554473777=39, srv1302117112=15, srv1322685828=17, srv1229105198=9, srv123897810=11, srv772633831=92, srv1259585224=13, srv1403066380=27, srv1603400360=42, srv1363386592=22, srv625314694=86, srv1767099283=51, srv1672132438=47, srv1936504850=61, srv945981964=99, srv1408834784=29} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-08 17:54:20,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-08 17:54:20,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-08 17:54:20,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-08 17:54:20,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-08 17:54:20,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-08 17:54:20,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-08 17:54:20,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-08 17:54:20,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-08 17:54:20,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-08 17:54:20,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-08 17:54:20,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-08 17:54:20,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-08 17:54:20,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-08 17:54:20,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-08 17:54:20,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-08 17:54:20,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-08 17:54:20,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-08 17:54:20,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-08 17:54:20,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-08 17:54:20,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-08 17:54:20,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-08 17:54:20,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-08 17:54:20,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-08 17:54:20,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-08 17:54:20,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-08 17:54:20,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-08 17:54:20,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-08 17:54:20,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-08 17:54:20,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-08 17:54:20,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-08 17:54:20,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-02-08 17:54:20,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-02-08 17:54:20,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-02-08 17:54:20,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-02-08 17:54:20,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-02-08 17:54:20,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-02-08 17:54:20,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-02-08 17:54:20,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-02-08 17:54:20,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-02-08 17:54:20,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-02-08 17:54:20,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-02-08 17:54:20,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-02-08 17:54:20,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-02-08 17:54:20,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-02-08 17:54:20,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-02-08 17:54:20,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-02-08 17:54:20,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-02-08 17:54:20,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-02-08 17:54:20,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-02-08 17:54:20,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-02-08 17:54:20,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-02-08 17:54:20,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-02-08 17:54:20,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-02-08 17:54:20,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-02-08 17:54:20,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-02-08 17:54:20,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-02-08 17:54:20,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-02-08 17:54:20,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-02-08 17:54:20,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-02-08 17:54:20,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-02-08 17:54:20,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-02-08 17:54:20,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-02-08 17:54:20,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-02-08 17:54:20,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-02-08 17:54:20,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-02-08 17:54:20,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-02-08 17:54:20,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-02-08 17:54:20,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-02-08 17:54:20,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-02-08 17:54:20,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-02-08 17:54:20,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-02-08 17:54:20,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-02-08 17:54:20,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-02-08 17:54:20,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-02-08 17:54:20,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-02-08 17:54:20,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-02-08 17:54:20,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-02-08 17:54:20,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-02-08 17:54:20,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-02-08 17:54:20,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-02-08 17:54:20,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-02-08 17:54:20,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-02-08 17:54:20,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-02-08 17:54:20,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-02-08 17:54:20,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-02-08 17:54:20,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-02-08 17:54:20,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-02-08 17:54:20,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-02-08 17:54:20,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-02-08 17:54:20,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-02-08 17:54:20,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-02-08 17:54:20,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-02-08 17:54:20,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-02-08 17:54:20,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-02-08 17:54:20,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-02-08 17:54:20,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-02-08 17:54:20,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-02-08 17:54:20,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-02-08 17:54:20,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-02-08 17:54:20,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-02-08 17:54:20,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-02-08 17:54:20,536 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.036764994167131866, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05320270197268334, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.036764994167131866, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17716666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24058333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-08 17:54:20,536 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-02-08 17:54:20,537 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18157467903225474, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.036764994167131866, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05320270197268334, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.036764994167131866, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17716666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24058333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-02-08 17:54:20,825 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 403 ms to try 5000 different iterations.  Found a solution that moves 2707 regions; Going from a computed imbalance of 0.18157467903225474 to a new imbalance of 0.014572548369803243. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02354958313206365); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03627430103840618, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.30077777777777776, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02354958313206365); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011583333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04275, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-08 17:54:20,836 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1914294513:77 , srv1650856945:80 , srv25395936:80 , srv1132191835:81 , srv1672132438:81 , srv681562098:81 , srv772633831:81 , srv1029743763:82 , srv125235427:82 , srv1268392819:82 , srv1311672063:82 , srv1322685828:82 , srv1341192625:82 , srv1358569171:82 , srv1453949934:82 , srv921885413:82 , srv1236295989:83 , srv1259585224:83 , srv1333246047:83 , srv1534175113:83 , srv1767099283:83 , srv1836653503:83 , srv1972806299:83 , srv811415096:83 , srv1205122848:84 , srv1229105198:84 , srv123897810:84 , srv1380037609:84 , srv1464045783:84 , srv1643705456:84 , srv181739199:84 , srv1870596302:84 , srv2075286304:84 , srv451015856:84 , srv4530785:84 , srv641731055:84 , srv919805028:84 , srv1363386592:85 , srv1375833022:85 , srv1445813830:85 , srv1491776671:85 , srv1566385257:85 , srv1615751628:85 , srv1839898159:85 , srv1942348226:85 , srv2005748246:85 , srv241838298:85 , srv364374670:85 , srv417264694:85 , srv667224089:85 , srv679782640:85 , srv1052697473:86 , srv1221758274:86 , srv1302117112:86 , srv139475738:86 , srv1408834784:86 , srv146325980:86 , srv1479091340:86 , srv1603400360:86 , srv1616613396:86 , srv1858397318:86 , srv1936504850:86 , srv2018602364:86 , srv2089081200:86 , srv259423077:86 , srv408911953:86 , srv538966467:86 , srv625314694:86 , srv877831921:86 , srv1033987955:87 , srv1198440602:87 , srv1399167940:87 , srv1403066380:87 , srv1403299137:87 , srv1554473777:87 , srv168447645:87 , srv1875195679:87 , srv1937498879:87 , srv2060552614:87 , srv3248236:87 , srv745375997:87 , srv1222392278:88 , srv1437178033:88 , srv1594463773:88 , srv1687847405:88 , srv1765082499:88 , srv1830584331:88 , srv1992580120:88 , srv2043081173:88 , srv241202465:88 , srv3371011:88 , srv441114406:88 , srv1351224093:89 , srv182771726:89 , srv868730494:89 , srv900965766:89 , srv194039828:90 , srv945981964:91 , srv1186022527:92 , srv1442351400:566 }
2024-02-08 17:54:20,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-08 17:54:20,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1268392819=14, srv1616613396=44, srv919805028=97, srv1442351400=31, srv1615751628=43, srv408911953=80, srv1445813830=32, srv364374670=79, srv1534175113=38, srv139475738=25, srv2060552614=70, srv1375833022=23, srv441114406=82, srv1222392278=8, srv1566385257=40, srv811415096=93, srv877831921=95, srv1033987955=1, srv1351224093=20, srv1236295989=10, srv241202465=73, srv2018602364=68, srv1830584331=54, srv1221758274=7, srv1870596302=58, srv679782640=89, srv194039828=63, srv868730494=94, srv2005748246=67, srv146325980=34, srv1132191835=3, srv3248236=77, srv1937498879=62, srv1437178033=30, srv1836653503=55, srv168447645=48, srv1358569171=21, srv4530785=84, srv1643705456=45, srv538966467=85, srv259423077=76, srv1491776671=37, srv1875195679=59, srv1914294513=60, srv125235427=12, srv1839898159=56, srv451015856=83, srv1341192625=19, srv417264694=81, srv1479091340=36, srv2075286304=71, srv1858397318=57, srv1186022527=4, srv25395936=75, srv1650856945=46, srv2043081173=69, srv1198440602=5, srv1029743763=0, srv182771726=53, srv681562098=90, srv1453949934=33, srv1942348226=64, srv1972806299=65, srv1403299137=28, srv1765082499=50, srv1687847405=49, srv921885413=98, srv1333246047=18, srv1380037609=24, srv1205122848=6, srv1311672063=16, srv1594463773=41, srv2089081200=72, srv3371011=78, srv1992580120=66, srv1464045783=35, srv745375997=91, srv181739199=52, srv641731055=87, srv241838298=74, srv1052697473=2, srv667224089=88, srv900965766=96, srv1399167940=26, srv1554473777=39, srv1302117112=15, srv1322685828=17, srv1229105198=9, srv123897810=11, srv772633831=92, srv1259585224=13, srv1403066380=27, srv1603400360=42, srv1363386592=22, srv625314694=86, srv1767099283=51, srv1672132438=47, srv1936504850=61, srv945981964=99, srv1408834784=29} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-08 17:54:20,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-08 17:54:20,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-08 17:54:20,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-08 17:54:20,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-08 17:54:20,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-08 17:54:20,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-08 17:54:20,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-08 17:54:20,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-08 17:54:20,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-08 17:54:20,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-08 17:54:20,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-08 17:54:20,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-08 17:54:20,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-08 17:54:20,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-08 17:54:20,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-08 17:54:20,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-08 17:54:20,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-08 17:54:20,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-08 17:54:20,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-08 17:54:20,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-08 17:54:20,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-08 17:54:20,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-08 17:54:20,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-08 17:54:20,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-08 17:54:20,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-08 17:54:20,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-08 17:54:20,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-08 17:54:20,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-08 17:54:20,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-08 17:54:20,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-08 17:54:20,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-02-08 17:54:20,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-02-08 17:54:20,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-02-08 17:54:20,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-02-08 17:54:20,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-02-08 17:54:20,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-02-08 17:54:20,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-02-08 17:54:20,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-02-08 17:54:20,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-02-08 17:54:20,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-02-08 17:54:20,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-02-08 17:54:20,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-02-08 17:54:20,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-02-08 17:54:20,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-02-08 17:54:20,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-02-08 17:54:20,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-02-08 17:54:20,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-02-08 17:54:20,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-02-08 17:54:20,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-02-08 17:54:20,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-02-08 17:54:20,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-02-08 17:54:20,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-02-08 17:54:20,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-02-08 17:54:20,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-02-08 17:54:20,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-02-08 17:54:20,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-02-08 17:54:20,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-02-08 17:54:20,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-02-08 17:54:20,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-02-08 17:54:20,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-02-08 17:54:20,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-02-08 17:54:20,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-02-08 17:54:20,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-02-08 17:54:20,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-02-08 17:54:20,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-02-08 17:54:20,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-02-08 17:54:20,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-02-08 17:54:20,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-02-08 17:54:20,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-02-08 17:54:20,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-02-08 17:54:20,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-02-08 17:54:20,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-02-08 17:54:20,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-02-08 17:54:20,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-02-08 17:54:20,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-02-08 17:54:20,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-02-08 17:54:20,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-02-08 17:54:20,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-02-08 17:54:20,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-02-08 17:54:20,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-02-08 17:54:20,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-02-08 17:54:20,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-02-08 17:54:20,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-02-08 17:54:20,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-02-08 17:54:20,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-02-08 17:54:20,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-02-08 17:54:20,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-02-08 17:54:20,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-02-08 17:54:20,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-02-08 17:54:20,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-02-08 17:54:20,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-02-08 17:54:20,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-02-08 17:54:20,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-02-08 17:54:20,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-02-08 17:54:20,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-02-08 17:54:20,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-02-08 17:54:20,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-02-08 17:54:20,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-02-08 17:54:20,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-02-08 17:54:20,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-02-08 17:54:20,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-02-08 17:54:21,000 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02354958313206365); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03627430103840618, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02354958313206365); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011583333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04275, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-08 17:54:21,001 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-02-08 17:54:21,001 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.01455359355067377, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02354958313206365); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03627430103840618, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02354958313206365); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011583333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04275, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-02-08 17:54:21,332 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 433 ms to try 5000 different iterations.  Found a solution that moves 940 regions; Going from a computed imbalance of 0.01455359355067377 to a new imbalance of 8.163304523681304E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006989918452319103); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009076307015852632); MoveCostFunction : (multiplier=7.0, imbalance=0.10444444444444445, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006989918452319103); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-08 17:54:21,334 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1186022527:79 , srv1403066380:80 , srv1351224093:82 , srv1858397318:82 , srv2089081200:82 , srv868730494:82 , srv1221758274:83 , srv1408834784:83 , srv1937498879:83 , srv194039828:83 , srv441114406:83 , srv538966467:83 , srv900965766:83 , srv945981964:83 , srv1341192625:84 , srv1437178033:84 , srv1594463773:84 , srv1603400360:84 , srv1942348226:84 , srv1992580120:84 , srv2043081173:84 , srv241202465:84 , srv25395936:84 , srv3248236:84 , srv3371011:84 , srv417264694:84 , srv745375997:84 , srv1375833022:85 , srv1399167940:85 , srv1403299137:85 , srv1445813830:85 , srv146325980:85 , srv1464045783:85 , srv1534175113:85 , srv1554473777:85 , srv1616613396:85 , srv1687847405:85 , srv1765082499:85 , srv182771726:85 , srv1830584331:85 , srv1836653503:85 , srv1839898159:85 , srv1875195679:85 , srv364374670:85 , srv625314694:85 , srv877831921:85 , srv1033987955:86 , srv1052697473:86 , srv1132191835:86 , srv1198440602:86 , srv1205122848:86 , srv1229105198:86 , srv1236295989:86 , srv1268392819:86 , srv1302117112:86 , srv1311672063:86 , srv1333246047:86 , srv139475738:86 , srv1672132438:86 , srv1936504850:86 , srv2018602364:86 , srv2060552614:86 , srv259423077:86 , srv408911953:86 , srv641731055:86 , srv667224089:86 , srv772633831:86 , srv1029743763:87 , srv1222392278:87 , srv125235427:87 , srv1358569171:87 , srv1363386592:87 , srv1380037609:87 , srv1479091340:87 , srv1491776671:87 , srv1566385257:87 , srv1615751628:87 , srv1767099283:87 , srv181739199:87 , srv2005748246:87 , srv2075286304:87 , srv241838298:87 , srv4530785:87 , srv679782640:87 , srv811415096:87 , srv919805028:87 , srv123897810:88 , srv1259585224:88 , srv1322685828:88 , srv1453949934:88 , srv1643705456:88 , srv168447645:88 , srv1870596302:88 , srv1914294513:88 , srv451015856:88 , srv681562098:88 , srv921885413:88 , srv1650856945:89 , srv1972806299:89 , srv1442351400:537 }
2024-02-08 17:54:21,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-08 17:54:21,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1268392819=14, srv1616613396=44, srv919805028=97, srv1442351400=31, srv1615751628=43, srv408911953=80, srv1445813830=32, srv364374670=79, srv1534175113=38, srv139475738=25, srv2060552614=70, srv1375833022=23, srv441114406=82, srv1222392278=8, srv1566385257=40, srv811415096=93, srv877831921=95, srv1033987955=1, srv1351224093=20, srv1236295989=10, srv241202465=73, srv2018602364=68, srv1830584331=54, srv1221758274=7, srv1870596302=58, srv679782640=89, srv194039828=63, srv868730494=94, srv2005748246=67, srv146325980=34, srv1132191835=3, srv3248236=77, srv1937498879=62, srv1437178033=30, srv1836653503=55, srv168447645=48, srv1358569171=21, srv4530785=84, srv1643705456=45, srv538966467=85, srv259423077=76, srv1491776671=37, srv1875195679=59, srv1914294513=60, srv125235427=12, srv1839898159=56, srv451015856=83, srv1341192625=19, srv417264694=81, srv1479091340=36, srv2075286304=71, srv1858397318=57, srv1186022527=4, srv25395936=75, srv1650856945=46, srv2043081173=69, srv1198440602=5, srv1029743763=0, srv182771726=53, srv681562098=90, srv1453949934=33, srv1942348226=64, srv1972806299=65, srv1403299137=28, srv1765082499=50, srv1687847405=49, srv921885413=98, srv1333246047=18, srv1380037609=24, srv1205122848=6, srv1311672063=16, srv1594463773=41, srv2089081200=72, srv3371011=78, srv1992580120=66, srv1464045783=35, srv745375997=91, srv181739199=52, srv641731055=87, srv241838298=74, srv1052697473=2, srv667224089=88, srv900965766=96, srv1399167940=26, srv1554473777=39, srv1302117112=15, srv1322685828=17, srv1229105198=9, srv123897810=11, srv772633831=92, srv1259585224=13, srv1403066380=27, srv1603400360=42, srv1363386592=22, srv625314694=86, srv1767099283=51, srv1672132438=47, srv1936504850=61, srv945981964=99, srv1408834784=29} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-08 17:54:21,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-08 17:54:21,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-08 17:54:21,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-08 17:54:21,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-08 17:54:21,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-08 17:54:21,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-08 17:54:21,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-08 17:54:21,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-08 17:54:21,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-08 17:54:21,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-08 17:54:21,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-08 17:54:21,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-08 17:54:21,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-08 17:54:21,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-08 17:54:21,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-08 17:54:21,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-08 17:54:21,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-08 17:54:21,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-08 17:54:21,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-08 17:54:21,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-08 17:54:21,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-08 17:54:21,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-08 17:54:21,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-08 17:54:21,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-08 17:54:21,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-08 17:54:21,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-08 17:54:21,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-08 17:54:21,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-08 17:54:21,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-08 17:54:21,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-08 17:54:21,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-02-08 17:54:21,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-02-08 17:54:21,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-02-08 17:54:21,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-02-08 17:54:21,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-02-08 17:54:21,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-02-08 17:54:21,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-02-08 17:54:21,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-02-08 17:54:21,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-02-08 17:54:21,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-02-08 17:54:21,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-02-08 17:54:21,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-02-08 17:54:21,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-02-08 17:54:21,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-02-08 17:54:21,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-02-08 17:54:21,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-02-08 17:54:21,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-02-08 17:54:21,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-02-08 17:54:21,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-02-08 17:54:21,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-02-08 17:54:21,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-02-08 17:54:21,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-02-08 17:54:21,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-02-08 17:54:21,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-02-08 17:54:21,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-02-08 17:54:21,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-02-08 17:54:21,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-02-08 17:54:21,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-02-08 17:54:21,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-02-08 17:54:21,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-02-08 17:54:21,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-02-08 17:54:21,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-02-08 17:54:21,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-02-08 17:54:21,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-02-08 17:54:21,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-02-08 17:54:21,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-02-08 17:54:21,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-02-08 17:54:21,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-02-08 17:54:21,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-02-08 17:54:21,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-02-08 17:54:21,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-02-08 17:54:21,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-02-08 17:54:21,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-02-08 17:54:21,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-02-08 17:54:21,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-02-08 17:54:21,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-02-08 17:54:21,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-02-08 17:54:21,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-02-08 17:54:21,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-02-08 17:54:21,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-02-08 17:54:21,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-02-08 17:54:21,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-02-08 17:54:21,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-02-08 17:54:21,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-02-08 17:54:21,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-02-08 17:54:21,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-02-08 17:54:21,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-02-08 17:54:21,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-02-08 17:54:21,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-02-08 17:54:21,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-02-08 17:54:21,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-02-08 17:54:21,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-02-08 17:54:21,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-02-08 17:54:21,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-02-08 17:54:21,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-02-08 17:54:21,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-02-08 17:54:21,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-02-08 17:54:21,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-02-08 17:54:21,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-02-08 17:54:21,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-02-08 17:54:21,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-02-08 17:54:21,460 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006989918452319103); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009076307015852632); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006989918452319103); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-08 17:54:21,461 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-02-08 17:54:21,461 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=8.097484316877815E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006989918452319103); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009076307015852632); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006989918452319103); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-02-08 17:54:21,601 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 233 ms to try 5000 different iterations.  Found a solution that moves 251 regions; Going from a computed imbalance of 8.097484316877815E-4 to a new imbalance of 1.35763859095521E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.23782800878924E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0020651563939657444); MoveCostFunction : (multiplier=7.0, imbalance=0.02788888888888889, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.23782800878924E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-08 17:54:21,601 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1221758274:83 , srv1222392278:83 , srv1554473777:83 , srv1914294513:83 , srv641731055:83 , srv1052697473:84 , srv1132191835:84 , srv1186022527:84 , srv1302117112:84 , srv1322685828:84 , srv1363386592:84 , srv1375833022:84 , srv1380037609:84 , srv139475738:84 , srv1399167940:84 , srv1403299137:84 , srv1437178033:84 , srv1453949934:84 , srv1616613396:84 , srv1643705456:84 , srv181739199:84 , srv182771726:84 , srv1870596302:84 , srv1875195679:84 , srv194039828:84 , srv1992580120:84 , srv2005748246:84 , srv2060552614:84 , srv2089081200:84 , srv3248236:84 , srv408911953:84 , srv451015856:84 , srv538966467:84 , srv679782640:84 , srv919805028:84 , srv1029743763:85 , srv1033987955:85 , srv1198440602:85 , srv1205122848:85 , srv1229105198:85 , srv1236295989:85 , srv123897810:85 , srv125235427:85 , srv1259585224:85 , srv1268392819:85 , srv1311672063:85 , srv1333246047:85 , srv1341192625:85 , srv1351224093:85 , srv1358569171:85 , srv1403066380:85 , srv1408834784:85 , srv1445813830:85 , srv146325980:85 , srv1479091340:85 , srv1491776671:85 , srv1534175113:85 , srv1566385257:85 , srv1594463773:85 , srv1603400360:85 , srv1615751628:85 , srv1650856945:85 , srv1672132438:85 , srv168447645:85 , srv1687847405:85 , srv1765082499:85 , srv1767099283:85 , srv1830584331:85 , srv1836653503:85 , srv1839898159:85 , srv1858397318:85 , srv1936504850:85 , srv1937498879:85 , srv1942348226:85 , srv1972806299:85 , srv2018602364:85 , srv2043081173:85 , srv2075286304:85 , srv241202465:85 , srv241838298:85 , srv25395936:85 , srv259423077:85 , srv3371011:85 , srv364374670:85 , srv417264694:85 , srv441114406:85 , srv4530785:85 , srv625314694:85 , srv681562098:85 , srv745375997:85 , srv772633831:85 , srv868730494:85 , srv877831921:85 , srv900965766:85 , srv921885413:85 , srv945981964:85 , srv1464045783:86 , srv667224089:86 , srv811415096:86 , srv1442351400:622 }
2024-02-08 17:54:21,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-08 17:54:21,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1268392819=14, srv1616613396=44, srv919805028=97, srv1442351400=31, srv1615751628=43, srv408911953=80, srv1445813830=32, srv364374670=79, srv1534175113=38, srv139475738=25, srv2060552614=70, srv1375833022=23, srv441114406=82, srv1222392278=8, srv1566385257=40, srv811415096=93, srv877831921=95, srv1033987955=1, srv1351224093=20, srv1236295989=10, srv241202465=73, srv2018602364=68, srv1830584331=54, srv1221758274=7, srv1870596302=58, srv679782640=89, srv194039828=63, srv868730494=94, srv2005748246=67, srv146325980=34, srv1132191835=3, srv3248236=77, srv1937498879=62, srv1437178033=30, srv1836653503=55, srv168447645=48, srv1358569171=21, srv4530785=84, srv1643705456=45, srv538966467=85, srv259423077=76, srv1491776671=37, srv1875195679=59, srv1914294513=60, srv125235427=12, srv1839898159=56, srv451015856=83, srv1341192625=19, srv417264694=81, srv1479091340=36, srv2075286304=71, srv1858397318=57, srv1186022527=4, srv25395936=75, srv1650856945=46, srv2043081173=69, srv1198440602=5, srv1029743763=0, srv182771726=53, srv681562098=90, srv1453949934=33, srv1942348226=64, srv1972806299=65, srv1403299137=28, srv1765082499=50, srv1687847405=49, srv921885413=98, srv1333246047=18, srv1380037609=24, srv1205122848=6, srv1311672063=16, srv1594463773=41, srv2089081200=72, srv3371011=78, srv1992580120=66, srv1464045783=35, srv745375997=91, srv181739199=52, srv641731055=87, srv241838298=74, srv1052697473=2, srv667224089=88, srv900965766=96, srv1399167940=26, srv1554473777=39, srv1302117112=15, srv1322685828=17, srv1229105198=9, srv123897810=11, srv772633831=92, srv1259585224=13, srv1403066380=27, srv1603400360=42, srv1363386592=22, srv625314694=86, srv1767099283=51, srv1672132438=47, srv1936504850=61, srv945981964=99, srv1408834784=29} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-08 17:54:21,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-08 17:54:21,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-08 17:54:21,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-08 17:54:21,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-08 17:54:21,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-08 17:54:21,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-08 17:54:21,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-08 17:54:21,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-08 17:54:21,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-08 17:54:21,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-08 17:54:21,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-08 17:54:21,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-08 17:54:21,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-08 17:54:21,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-08 17:54:21,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-08 17:54:21,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-08 17:54:21,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-08 17:54:21,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-08 17:54:21,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-08 17:54:21,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-08 17:54:21,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-08 17:54:21,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-08 17:54:21,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-08 17:54:21,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-08 17:54:21,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-08 17:54:21,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-08 17:54:21,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-08 17:54:21,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-08 17:54:21,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-08 17:54:21,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-08 17:54:21,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-02-08 17:54:21,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-02-08 17:54:21,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-02-08 17:54:21,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-02-08 17:54:21,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-02-08 17:54:21,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-02-08 17:54:21,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-02-08 17:54:21,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-02-08 17:54:21,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-02-08 17:54:21,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-02-08 17:54:21,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-02-08 17:54:21,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-02-08 17:54:21,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-02-08 17:54:21,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-02-08 17:54:21,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-02-08 17:54:21,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-02-08 17:54:21,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-02-08 17:54:21,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-02-08 17:54:21,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-02-08 17:54:21,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-02-08 17:54:21,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-02-08 17:54:21,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-02-08 17:54:21,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-02-08 17:54:21,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-02-08 17:54:21,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-02-08 17:54:21,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-02-08 17:54:21,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-02-08 17:54:21,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-02-08 17:54:21,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-02-08 17:54:21,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-02-08 17:54:21,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-02-08 17:54:21,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-02-08 17:54:21,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-02-08 17:54:21,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-02-08 17:54:21,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-02-08 17:54:21,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-02-08 17:54:21,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-02-08 17:54:21,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-02-08 17:54:21,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-02-08 17:54:21,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-02-08 17:54:21,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-02-08 17:54:21,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-02-08 17:54:21,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-02-08 17:54:21,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-02-08 17:54:21,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-02-08 17:54:21,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-02-08 17:54:21,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-02-08 17:54:21,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-02-08 17:54:21,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-02-08 17:54:21,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-02-08 17:54:21,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-02-08 17:54:21,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-02-08 17:54:21,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-02-08 17:54:21,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-02-08 17:54:21,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-02-08 17:54:21,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-02-08 17:54:21,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-02-08 17:54:21,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-02-08 17:54:21,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-02-08 17:54:21,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-02-08 17:54:21,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-02-08 17:54:21,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-02-08 17:54:21,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-02-08 17:54:21,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-02-08 17:54:21,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-02-08 17:54:21,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-02-08 17:54:21,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-02-08 17:54:21,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-02-08 17:54:21,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-02-08 17:54:21,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-02-08 17:54:21,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-02-08 17:54:21,703 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.1818846344905755E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.23782800878924E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0020651563939657444); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.23782800878924E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-08 17:54:21,703 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1221758274:83 , srv1222392278:83 , srv1554473777:83 , srv1914294513:83 , srv641731055:83 , srv1052697473:84 , srv1132191835:84 , srv1186022527:84 , srv1302117112:84 , srv1322685828:84 , srv1363386592:84 , srv1375833022:84 , srv1380037609:84 , srv139475738:84 , srv1399167940:84 , srv1403299137:84 , srv1437178033:84 , srv1453949934:84 , srv1616613396:84 , srv1643705456:84 , srv181739199:84 , srv182771726:84 , srv1870596302:84 , srv1875195679:84 , srv194039828:84 , srv1992580120:84 , srv2005748246:84 , srv2060552614:84 , srv2089081200:84 , srv3248236:84 , srv408911953:84 , srv451015856:84 , srv538966467:84 , srv679782640:84 , srv919805028:84 , srv1029743763:85 , srv1033987955:85 , srv1198440602:85 , srv1205122848:85 , srv1229105198:85 , srv1236295989:85 , srv123897810:85 , srv125235427:85 , srv1259585224:85 , srv1268392819:85 , srv1311672063:85 , srv1333246047:85 , srv1341192625:85 , srv1351224093:85 , srv1358569171:85 , srv1403066380:85 , srv1408834784:85 , srv1445813830:85 , srv146325980:85 , srv1479091340:85 , srv1491776671:85 , srv1534175113:85 , srv1566385257:85 , srv1594463773:85 , srv1603400360:85 , srv1615751628:85 , srv1650856945:85 , srv1672132438:85 , srv168447645:85 , srv1687847405:85 , srv1765082499:85 , srv1767099283:85 , srv1830584331:85 , srv1836653503:85 , srv1839898159:85 , srv1858397318:85 , srv1936504850:85 , srv1937498879:85 , srv1942348226:85 , srv1972806299:85 , srv2018602364:85 , srv2043081173:85 , srv2075286304:85 , srv241202465:85 , srv241838298:85 , srv25395936:85 , srv259423077:85 , srv3371011:85 , srv364374670:85 , srv417264694:85 , srv441114406:85 , srv4530785:85 , srv625314694:85 , srv681562098:85 , srv745375997:85 , srv772633831:85 , srv868730494:85 , srv877831921:85 , srv900965766:85 , srv921885413:85 , srv945981964:85 , srv1464045783:86 , srv667224089:86 , srv811415096:86 , srv1442351400:622 }
2024-02-08 17:54:21,730 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=269 (was 269), ProcessCount=175 (was 175), AvailableMemoryMB=1588 (was 2219)
2024-02-08 17:54:21,741 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=269, ProcessCount=175, AvailableMemoryMB=1587
2024-02-08 17:54:21,742 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-02-08 17:54:21,742 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2024-02-08 17:54:21,749 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv104988205:84 , srv1090439161:84 , srv1206931329:84 , srv1214098745:84 , srv1318173457:84 , srv1324507182:84 , srv1457702450:84 , srv1517492511:84 , srv1533656528:84 , srv1561394066:84 , srv1596649621:84 , srv1724521112:84 , srv1830270332:84 , srv1867951015:84 , srv1904629570:84 , srv1927759226:84 , srv1981436580:84 , srv1982292549:84 , srv1982450928:84 , srv2052426053:84 , srv235853358:84 , srv584240797:84 , srv589888662:84 , srv601938423:84 , srv623811211:84 , srv744557249:84 , srv801403992:84 , srv914059253:84 , srv955907856:84 , srv273146553:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2024-02-08 17:54:21,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-08 17:54:21,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1533656528=8, srv623811211=25, srv914059253=28, srv1090439161=1, srv235853358=20, srv744557249=26, srv1324507182=5, srv1517492511=7, srv1982450928=18, srv2052426053=19, srv601938423=24, srv1214098745=3, srv1724521112=11, srv1561394066=9, srv273146553=21, srv1867951015=13, srv1982292549=17, srv955907856=29, srv1206931329=2, srv1927759226=15, srv801403992=27, srv1981436580=16, srv1318173457=4, srv1457702450=6, srv104988205=0, srv584240797=22, srv589888662=23, srv1830270332=12, srv1596649621=10, srv1904629570=14} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-08 17:54:21,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-08 17:54:21,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-08 17:54:21,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-08 17:54:21,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-08 17:54:21,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-08 17:54:21,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-08 17:54:21,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-08 17:54:21,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-08 17:54:21,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-08 17:54:21,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-08 17:54:21,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-08 17:54:21,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-08 17:54:21,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-08 17:54:21,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-08 17:54:21,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-08 17:54:21,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-08 17:54:21,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-08 17:54:21,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-08 17:54:21,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-08 17:54:21,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-08 17:54:21,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-08 17:54:21,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-08 17:54:21,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-08 17:54:21,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-08 17:54:21,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-08 17:54:21,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-08 17:54:21,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-08 17:54:21,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-08 17:54:21,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-08 17:54:21,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-08 17:54:21,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-02-08 17:54:21,772 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1189481947386841, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-08 17:54:21,773 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912567845141885, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1189481947386841, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2024-02-08 17:56:21,762 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 11044121 different iterations.  Found a solution that moves 1838 regions; Going from a computed imbalance of 0.9912567845141885 to a new imbalance of 4.2899836916600065E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6807407407407408, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-08 17:56:21,767 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv104988205:90 , srv1090439161:90 , srv1206931329:90 , srv1214098745:90 , srv1318173457:90 , srv1324507182:90 , srv1457702450:90 , srv1517492511:90 , srv1533656528:90 , srv1561394066:90 , srv1596649621:90 , srv1724521112:90 , srv1830270332:90 , srv1867951015:90 , srv1904629570:90 , srv1927759226:90 , srv1981436580:90 , srv1982292549:90 , srv1982450928:90 , srv2052426053:90 , srv235853358:90 , srv273146553:90 , srv584240797:90 , srv589888662:90 , srv601938423:90 , srv623811211:90 , srv744557249:90 , srv801403992:90 , srv914059253:90 , srv955907856:90 }
2024-02-08 17:56:21,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-08 17:56:21,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1533656528=8, srv623811211=25, srv914059253=28, srv1090439161=1, srv235853358=20, srv744557249=26, srv1324507182=5, srv1517492511=7, srv1982450928=18, srv2052426053=19, srv601938423=24, srv1214098745=3, srv1724521112=11, srv1561394066=9, srv273146553=21, srv1867951015=13, srv1982292549=17, srv955907856=29, srv1206931329=2, srv1927759226=15, srv801403992=27, srv1981436580=16, srv1318173457=4, srv1457702450=6, srv104988205=0, srv584240797=22, srv589888662=23, srv1830270332=12, srv1596649621=10, srv1904629570=14} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-08 17:56:21,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-08 17:56:21,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-08 17:56:21,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-08 17:56:21,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-08 17:56:21,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-08 17:56:21,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-08 17:56:21,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-08 17:56:21,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-08 17:56:21,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-08 17:56:21,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-08 17:56:21,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-08 17:56:21,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-08 17:56:21,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-08 17:56:21,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-08 17:56:21,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-08 17:56:21,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-08 17:56:21,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-08 17:56:21,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-08 17:56:21,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-08 17:56:21,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-08 17:56:21,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-08 17:56:21,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-08 17:56:21,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-08 17:56:21,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-08 17:56:21,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-08 17:56:21,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-08 17:56:21,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-08 17:56:21,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-08 17:56:21,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-08 17:56:21,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-08 17:56:21,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-02-08 17:56:21,781 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-08 17:56:21,781 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv104988205:90 , srv1090439161:90 , srv1206931329:90 , srv1214098745:90 , srv1318173457:90 , srv1324507182:90 , srv1457702450:90 , srv1517492511:90 , srv1533656528:90 , srv1561394066:90 , srv1596649621:90 , srv1724521112:90 , srv1830270332:90 , srv1867951015:90 , srv1904629570:90 , srv1927759226:90 , srv1981436580:90 , srv1982292549:90 , srv1982450928:90 , srv2052426053:90 , srv235853358:90 , srv273146553:90 , srv584240797:90 , srv589888662:90 , srv601938423:90 , srv623811211:90 , srv744557249:90 , srv801403992:90 , srv914059253:90 , srv955907856:90 }
2024-02-08 17:56:21,793 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=134 (was 269), ProcessCount=173 (was 175), AvailableMemoryMB=2781 (was 1587) - AvailableMemoryMB LEAK? -