2023-12-09 01:56:45,897 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-12-09 01:56:46,192 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-12-09 01:56:46,336 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-12-09 01:56:46,406 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-12-09 01:56:46,413 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-12-09 01:56:46,425 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=284, ProcessCount=172, AvailableMemoryMB=2856
2023-12-09 01:56:46,427 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-12-09 01:56:46,428 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-12-09 01:56:46,612 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1014734461:84 , srv1033261441:84 , srv1062764260:84 , srv1092333729:84 , srv109507858:84 , srv1103992068:84 , srv1108382202:84 , srv1112584644:84 , srv1117142075:84 , srv1122904371:84 , srv1129291300:84 , srv1131421187:84 , srv1149607492:84 , srv1153553428:84 , srv1164938172:84 , srv116543514:84 , srv1191155661:84 , srv1209040471:84 , srv1218169184:84 , srv1265713700:84 , srv1283850630:84 , srv1319189526:84 , srv1334923670:84 , srv1354777676:84 , srv1388078833:84 , srv1397892632:84 , srv1430884532:84 , srv1444961641:84 , srv1444974627:84 , srv1458870020:84 , srv1466735939:84 , srv146980876:84 , srv1470729187:84 , srv1471919441:84 , srv1540841235:84 , srv1551364951:84 , srv1559294963:84 , srv1584598837:84 , srv1675163861:84 , srv1695057258:84 , srv1698313941:84 , srv1706339786:84 , srv1733131751:84 , srv1766688890:84 , srv1774358642:84 , srv1786970959:84 , srv1819548513:84 , srv1822980460:84 , srv1846985547:84 , srv1855568756:84 , srv1860973999:84 , srv1861311921:84 , srv1890281929:84 , srv1932338355:84 , srv1937557087:84 , srv1938861902:84 , srv1961366229:84 , srv2043993912:84 , srv2076707974:84 , srv2099156144:84 , srv2117681265:84 , srv2122192450:84 , srv229904173:84 , srv23115946:84 , srv238641144:84 , srv261292142:84 , srv297524976:84 , srv305117158:84 , srv327558961:84 , srv357010475:84 , srv422895581:84 , srv42771606:84 , srv440425570:84 , srv446446669:84 , srv491540662:84 , srv498405556:84 , srv52967638:84 , srv568869182:84 , srv570707944:84 , srv581240746:84 , srv606768633:84 , srv615946819:84 , srv634112752:84 , srv656477577:84 , srv680294968:84 , srv699244241:84 , srv765281136:84 , srv780335142:84 , srv809833735:84 , srv825741765:84 , srv86425939:84 , srv912146513:84 , srv915404727:84 , srv930478406:84 , srv939171106:84 , srv940045809:84 , srv944552158:84 , srv971080517:84 , srv990896250:84 , srv1295779276:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-12-09 01:56:46,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-09 01:56:46,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1706339786=42, srv1444974627=29, srv116543514=15, srv990896250=99, srv1153553428=13, srv1846985547=49, srv680294968=85, srv825741765=90, srv940045809=96, srv1218169184=18, srv1283850630=20, srv1584598837=38, srv1540841235=35, srv305117158=68, srv780335142=88, srv52967638=77, srv581240746=80, srv1695057258=40, srv912146513=92, srv1466735939=31, srv971080517=98, srv1822980460=48, srv238641144=65, srv915404727=93, srv297524976=67, srv1786970959=46, srv1733131751=43, srv357010475=70, srv446446669=74, srv606768633=81, srv1861311921=52, srv422895581=71, srv568869182=78, srv699244241=86, srv1112584644=7, srv930478406=94, srv491540662=75, srv42771606=72, srv1397892632=26, srv2043993912=58, srv440425570=73, srv634112752=83, srv1470729187=33, srv1471919441=34, srv1766688890=44, srv109507858=4, srv1062764260=2, srv23115946=64, srv1458870020=30, srv1265713700=19, srv1430884532=27, srv1774358642=45, srv1937557087=55, srv1129291300=10, srv1117142075=8, srv1961366229=57, srv1295779276=21, srv86425939=91, srv261292142=66, srv2099156144=60, srv1103992068=5, srv615946819=82, srv1209040471=17, srv1675163861=39, srv2122192450=62, srv1819548513=47, srv1938861902=56, srv765281136=87, srv1014734461=0, srv1319189526=22, srv2076707974=59, srv327558961=69, srv1164938172=14, srv1122904371=9, srv1033261441=1, srv656477577=84, srv570707944=79, srv1334923670=23, srv809833735=89, srv939171106=95, srv1559294963=37, srv1932338355=54, srv229904173=63, srv1855568756=50, srv1860973999=51, srv2117681265=61, srv1698313941=41, srv944552158=97, srv1108382202=6, srv1131421187=11, srv1551364951=36, srv1890281929=53, srv1149607492=12, srv1354777676=24, srv498405556=76, srv1191155661=16, srv1444961641=28, srv1092333729=3, srv146980876=32, srv1388078833=25} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-09 01:56:46,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-09 01:56:46,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-09 01:56:46,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-09 01:56:46,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-09 01:56:46,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-09 01:56:46,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-09 01:56:46,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-09 01:56:46,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-09 01:56:46,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-09 01:56:46,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-09 01:56:46,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-09 01:56:46,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-09 01:56:46,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-09 01:56:46,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-09 01:56:46,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-09 01:56:46,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-09 01:56:46,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-09 01:56:46,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-09 01:56:46,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-09 01:56:46,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-09 01:56:46,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-09 01:56:46,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-09 01:56:46,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-09 01:56:46,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-09 01:56:46,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-09 01:56:46,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-09 01:56:46,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-09 01:56:46,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-09 01:56:46,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-09 01:56:46,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-09 01:56:46,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-09 01:56:46,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-09 01:56:46,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-09 01:56:46,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-09 01:56:46,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-09 01:56:46,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-09 01:56:46,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-09 01:56:46,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-09 01:56:46,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-09 01:56:46,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-09 01:56:46,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-09 01:56:46,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-09 01:56:46,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-09 01:56:46,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-09 01:56:46,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-09 01:56:46,717 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-09 01:56:46,717 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-09 01:56:46,717 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-09 01:56:46,717 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-09 01:56:46,717 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-09 01:56:46,717 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-09 01:56:46,717 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-09 01:56:46,717 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-09 01:56:46,717 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-09 01:56:46,717 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-09 01:56:46,717 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-09 01:56:46,718 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-09 01:56:46,718 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-09 01:56:46,718 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-09 01:56:46,718 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-09 01:56:46,718 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-09 01:56:46,718 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-09 01:56:46,718 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-09 01:56:46,718 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-09 01:56:46,718 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-09 01:56:46,718 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-09 01:56:46,718 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-09 01:56:46,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-09 01:56:46,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-09 01:56:46,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-09 01:56:46,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-09 01:56:46,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-09 01:56:46,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-09 01:56:46,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-09 01:56:46,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-09 01:56:46,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-09 01:56:46,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-09 01:56:46,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-09 01:56:46,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-09 01:56:46,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-09 01:56:46,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-09 01:56:46,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-09 01:56:46,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-09 01:56:46,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-09 01:56:46,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-09 01:56:46,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-09 01:56:46,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-09 01:56:46,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-09 01:56:46,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-09 01:56:46,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-09 01:56:46,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-09 01:56:46,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-09 01:56:46,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-09 01:56:46,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-09 01:56:46,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-09 01:56:46,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-09 01:56:46,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-09 01:56:46,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-09 01:56:46,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-09 01:56:46,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-09 01:56:46,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-09 01:56:47,069 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-09 01:56:47,070 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-12-09 01:56:47,070 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-12-09 01:56:47,526 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 782 ms to try 5000 different iterations.  Found a solution that moves 4717 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17568222162226618. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03865264139252684, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.053145713760693415, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5241111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03865264139252684, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17091666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23716666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-09 01:56:47,539 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1354777676:80 , srv1733131751:80 , srv912146513:81 , srv116543514:82 , srv1397892632:82 , srv1471919441:82 , srv2043993912:82 , srv238641144:82 , srv42771606:82 , srv1092333729:83 , srv1551364951:83 , srv1766688890:83 , srv1937557087:83 , srv357010475:83 , srv656477577:83 , srv930478406:83 , srv1112584644:84 , srv1129291300:84 , srv1209040471:84 , srv1458870020:84 , srv491540662:84 , srv570707944:84 , srv680294968:84 , srv86425939:84 , srv1108382202:85 , srv1164938172:85 , srv1191155661:85 , srv1218169184:85 , srv1470729187:85 , srv1559294963:85 , srv1890281929:85 , srv2076707974:85 , srv2099156144:85 , srv305117158:85 , srv440425570:85 , srv52967638:85 , srv1062764260:86 , srv109507858:86 , srv1117142075:86 , srv1131421187:86 , srv1540841235:86 , srv1584598837:86 , srv1698313941:86 , srv1786970959:86 , srv1855568756:86 , srv1860973999:86 , srv1938861902:86 , srv498405556:86 , srv568869182:86 , srv606768633:86 , srv699244241:86 , srv765281136:86 , srv780335142:86 , srv1014734461:87 , srv1149607492:87 , srv1444974627:87 , srv1932338355:87 , srv1961366229:87 , srv2122192450:87 , srv23115946:87 , srv261292142:87 , srv327558961:87 , srv422895581:87 , srv825741765:87 , srv971080517:87 , srv1033261441:88 , srv1265713700:88 , srv1283850630:88 , srv1388078833:88 , srv1444961641:88 , srv1466735939:88 , srv1695057258:88 , srv1861311921:88 , srv297524976:88 , srv446446669:88 , srv939171106:88 , srv940045809:88 , srv944552158:88 , srv146980876:89 , srv1675163861:89 , srv1774358642:89 , srv2117681265:89 , srv581240746:89 , srv634112752:89 , srv809833735:89 , srv1103992068:90 , srv1819548513:90 , srv1846985547:90 , srv229904173:90 , srv1334923670:91 , srv1430884532:91 , srv1822980460:91 , srv615946819:91 , srv915404727:91 , srv1153553428:92 , srv1319189526:94 , srv990896250:95 , srv1122904371:96 , srv1706339786:96 , srv1295779276:433 }
2023-12-09 01:56:47,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-09 01:56:47,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1706339786=42, srv1444974627=29, srv116543514=15, srv990896250=99, srv1153553428=13, srv1846985547=49, srv680294968=85, srv825741765=90, srv940045809=96, srv1218169184=18, srv1283850630=20, srv1584598837=38, srv1540841235=35, srv305117158=68, srv780335142=88, srv52967638=77, srv581240746=80, srv1695057258=40, srv912146513=92, srv1466735939=31, srv971080517=98, srv1822980460=48, srv238641144=65, srv915404727=93, srv297524976=67, srv1786970959=46, srv1733131751=43, srv357010475=70, srv446446669=74, srv606768633=81, srv1861311921=52, srv422895581=71, srv568869182=78, srv699244241=86, srv1112584644=7, srv930478406=94, srv491540662=75, srv42771606=72, srv1397892632=26, srv2043993912=58, srv440425570=73, srv634112752=83, srv1470729187=33, srv1471919441=34, srv1766688890=44, srv109507858=4, srv1062764260=2, srv23115946=64, srv1458870020=30, srv1265713700=19, srv1430884532=27, srv1774358642=45, srv1937557087=55, srv1129291300=10, srv1117142075=8, srv1961366229=57, srv1295779276=21, srv86425939=91, srv261292142=66, srv2099156144=60, srv1103992068=5, srv615946819=82, srv1209040471=17, srv1675163861=39, srv2122192450=62, srv1819548513=47, srv1938861902=56, srv765281136=87, srv1014734461=0, srv1319189526=22, srv2076707974=59, srv327558961=69, srv1164938172=14, srv1122904371=9, srv1033261441=1, srv656477577=84, srv570707944=79, srv1334923670=23, srv809833735=89, srv939171106=95, srv1559294963=37, srv1932338355=54, srv229904173=63, srv1855568756=50, srv1860973999=51, srv2117681265=61, srv1698313941=41, srv944552158=97, srv1108382202=6, srv1131421187=11, srv1551364951=36, srv1890281929=53, srv1149607492=12, srv1354777676=24, srv498405556=76, srv1191155661=16, srv1444961641=28, srv1092333729=3, srv146980876=32, srv1388078833=25} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-09 01:56:47,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-09 01:56:47,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-09 01:56:47,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-09 01:56:47,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-09 01:56:47,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-09 01:56:47,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-09 01:56:47,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-09 01:56:47,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-09 01:56:47,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-09 01:56:47,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-09 01:56:47,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-09 01:56:47,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-09 01:56:47,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-09 01:56:47,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-09 01:56:47,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-09 01:56:47,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-09 01:56:47,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-09 01:56:47,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-09 01:56:47,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-09 01:56:47,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-09 01:56:47,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-09 01:56:47,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-09 01:56:47,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-09 01:56:47,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-09 01:56:47,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-09 01:56:47,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-09 01:56:47,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-09 01:56:47,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-09 01:56:47,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-09 01:56:47,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-09 01:56:47,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-09 01:56:47,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-09 01:56:47,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-09 01:56:47,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-09 01:56:47,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-09 01:56:47,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-09 01:56:47,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-09 01:56:47,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-09 01:56:47,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-09 01:56:47,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-09 01:56:47,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-09 01:56:47,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-09 01:56:47,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-09 01:56:47,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-09 01:56:47,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-09 01:56:47,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-09 01:56:47,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-09 01:56:47,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-09 01:56:47,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-09 01:56:47,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-09 01:56:47,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-09 01:56:47,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-09 01:56:47,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-09 01:56:47,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-09 01:56:47,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-09 01:56:47,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-09 01:56:47,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-09 01:56:47,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-09 01:56:47,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-09 01:56:47,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-09 01:56:47,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-09 01:56:47,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-09 01:56:47,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-09 01:56:47,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-09 01:56:47,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-09 01:56:47,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-09 01:56:47,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-09 01:56:47,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-09 01:56:47,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-09 01:56:47,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-09 01:56:47,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-09 01:56:47,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-09 01:56:47,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-09 01:56:47,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-09 01:56:47,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-09 01:56:47,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-09 01:56:47,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-09 01:56:47,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-09 01:56:47,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-09 01:56:47,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-09 01:56:47,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-09 01:56:47,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-09 01:56:47,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-09 01:56:47,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-09 01:56:47,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-09 01:56:47,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-09 01:56:47,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-09 01:56:47,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-09 01:56:47,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-09 01:56:47,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-09 01:56:47,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-09 01:56:47,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-09 01:56:47,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-09 01:56:47,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-09 01:56:47,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-09 01:56:47,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-09 01:56:47,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-09 01:56:47,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-09 01:56:47,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-09 01:56:47,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-09 01:56:47,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-09 01:56:47,712 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03865264139252684, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.053145713760693415, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03865264139252684, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17091666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23716666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-09 01:56:47,712 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-12-09 01:56:47,712 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17564919248232025, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03865264139252684, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.053145713760693415, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03865264139252684, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17091666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23716666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-12-09 01:56:47,912 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 310 ms to try 5000 different iterations.  Found a solution that moves 2675 regions; Going from a computed imbalance of 0.17564919248232025 to a new imbalance of 0.014117982549016653. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.019916899310996165); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.027560768487035328, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2972222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.019916899310996165); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011166666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0425, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-09 01:56:47,932 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1122904371:76 , srv990896250:76 , srv1706339786:78 , srv615946819:80 , srv1103992068:81 , srv1319189526:81 , srv1822980460:81 , srv229904173:81 , srv915404727:81 , srv1033261441:82 , srv1153553428:82 , srv146980876:82 , srv1846985547:82 , srv2117681265:82 , srv581240746:82 , srv634112752:82 , srv1334923670:83 , srv1430884532:83 , srv1819548513:83 , srv1961366229:83 , srv446446669:83 , srv1265713700:84 , srv1283850630:84 , srv1675163861:84 , srv1695057258:84 , srv1774358642:84 , srv1861311921:84 , srv297524976:84 , srv327558961:84 , srv809833735:84 , srv825741765:84 , srv944552158:84 , srv971080517:84 , srv1014734461:85 , srv1388078833:85 , srv1444961641:85 , srv1466735939:85 , srv1540841235:85 , srv1584598837:85 , srv23115946:85 , srv498405556:85 , srv52967638:85 , srv699244241:85 , srv765281136:85 , srv780335142:85 , srv940045809:85 , srv1108382202:86 , srv1131421187:86 , srv1149607492:86 , srv1191155661:86 , srv1218169184:86 , srv1698313941:86 , srv1786970959:86 , srv1855568756:86 , srv1860973999:86 , srv2122192450:86 , srv261292142:86 , srv422895581:86 , srv568869182:86 , srv570707944:86 , srv606768633:86 , srv1062764260:87 , srv109507858:87 , srv1112584644:87 , srv1117142075:87 , srv1164938172:87 , srv1209040471:87 , srv1444974627:87 , srv1470729187:87 , srv1932338355:87 , srv305117158:87 , srv440425570:87 , srv680294968:87 , srv939171106:87 , srv116543514:88 , srv1458870020:88 , srv1551364951:88 , srv1559294963:88 , srv1938861902:88 , srv2076707974:88 , srv1092333729:89 , srv1129291300:89 , srv1471919441:89 , srv1890281929:89 , srv1937557087:89 , srv357010475:89 , srv491540662:89 , srv86425939:89 , srv1397892632:90 , srv238641144:90 , srv42771606:90 , srv656477577:90 , srv930478406:90 , srv1733131751:91 , srv1766688890:91 , srv2043993912:91 , srv2099156144:92 , srv1354777676:94 , srv912146513:95 , srv1295779276:518 }
2023-12-09 01:56:47,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-09 01:56:47,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1706339786=42, srv1444974627=29, srv116543514=15, srv990896250=99, srv1153553428=13, srv1846985547=49, srv680294968=85, srv825741765=90, srv940045809=96, srv1218169184=18, srv1283850630=20, srv1584598837=38, srv1540841235=35, srv305117158=68, srv780335142=88, srv52967638=77, srv581240746=80, srv1695057258=40, srv912146513=92, srv1466735939=31, srv971080517=98, srv1822980460=48, srv238641144=65, srv915404727=93, srv297524976=67, srv1786970959=46, srv1733131751=43, srv357010475=70, srv446446669=74, srv606768633=81, srv1861311921=52, srv422895581=71, srv568869182=78, srv699244241=86, srv1112584644=7, srv930478406=94, srv491540662=75, srv42771606=72, srv1397892632=26, srv2043993912=58, srv440425570=73, srv634112752=83, srv1470729187=33, srv1471919441=34, srv1766688890=44, srv109507858=4, srv1062764260=2, srv23115946=64, srv1458870020=30, srv1265713700=19, srv1430884532=27, srv1774358642=45, srv1937557087=55, srv1129291300=10, srv1117142075=8, srv1961366229=57, srv1295779276=21, srv86425939=91, srv261292142=66, srv2099156144=60, srv1103992068=5, srv615946819=82, srv1209040471=17, srv1675163861=39, srv2122192450=62, srv1819548513=47, srv1938861902=56, srv765281136=87, srv1014734461=0, srv1319189526=22, srv2076707974=59, srv327558961=69, srv1164938172=14, srv1122904371=9, srv1033261441=1, srv656477577=84, srv570707944=79, srv1334923670=23, srv809833735=89, srv939171106=95, srv1559294963=37, srv1932338355=54, srv229904173=63, srv1855568756=50, srv1860973999=51, srv2117681265=61, srv1698313941=41, srv944552158=97, srv1108382202=6, srv1131421187=11, srv1551364951=36, srv1890281929=53, srv1149607492=12, srv1354777676=24, srv498405556=76, srv1191155661=16, srv1444961641=28, srv1092333729=3, srv146980876=32, srv1388078833=25} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-09 01:56:47,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-09 01:56:47,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-09 01:56:47,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-09 01:56:47,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-09 01:56:47,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-09 01:56:47,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-09 01:56:47,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-09 01:56:47,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-09 01:56:47,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-09 01:56:47,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-09 01:56:47,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-09 01:56:47,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-09 01:56:47,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-09 01:56:47,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-09 01:56:47,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-09 01:56:47,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-09 01:56:47,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-09 01:56:47,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-09 01:56:47,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-09 01:56:47,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-09 01:56:47,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-09 01:56:47,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-09 01:56:47,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-09 01:56:47,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-09 01:56:47,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-09 01:56:47,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-09 01:56:47,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-09 01:56:47,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-09 01:56:47,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-09 01:56:47,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-09 01:56:47,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-09 01:56:47,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-09 01:56:47,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-09 01:56:47,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-09 01:56:47,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-09 01:56:47,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-09 01:56:47,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-09 01:56:47,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-09 01:56:47,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-09 01:56:47,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-09 01:56:47,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-09 01:56:47,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-09 01:56:47,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-09 01:56:47,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-09 01:56:47,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-09 01:56:47,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-09 01:56:47,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-09 01:56:47,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-09 01:56:47,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-09 01:56:47,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-09 01:56:47,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-09 01:56:47,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-09 01:56:47,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-09 01:56:47,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-09 01:56:47,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-09 01:56:47,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-09 01:56:47,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-09 01:56:47,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-09 01:56:47,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-09 01:56:47,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-09 01:56:47,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-09 01:56:47,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-09 01:56:47,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-09 01:56:47,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-09 01:56:47,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-09 01:56:47,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-09 01:56:47,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-09 01:56:47,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-09 01:56:47,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-09 01:56:47,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-09 01:56:47,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-09 01:56:47,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-09 01:56:47,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-09 01:56:47,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-09 01:56:47,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-09 01:56:47,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-09 01:56:47,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-09 01:56:47,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-09 01:56:47,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-09 01:56:47,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-09 01:56:47,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-09 01:56:47,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-09 01:56:47,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-09 01:56:47,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-09 01:56:47,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-09 01:56:47,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-09 01:56:47,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-09 01:56:47,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-09 01:56:47,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-09 01:56:47,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-09 01:56:47,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-09 01:56:47,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-09 01:56:47,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-09 01:56:47,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-09 01:56:47,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-09 01:56:47,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-09 01:56:47,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-09 01:56:47,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-09 01:56:47,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-09 01:56:47,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-09 01:56:47,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-09 01:56:48,057 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.019916899310996165); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.027560768487035328, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.019916899310996165); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011166666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0425, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-09 01:56:48,057 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-12-09 01:56:48,057 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.014099251798676299, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.019916899310996165); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.027560768487035328, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.019916899310996165); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011166666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0425, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-12-09 01:56:48,216 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 240 ms to try 5000 different iterations.  Found a solution that moves 1054 regions; Going from a computed imbalance of 0.014099251798676299 to a new imbalance of 2.7424820332874386E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005549941046232873); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005014009553935699); MoveCostFunction : (multiplier=7.0, imbalance=0.11711111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005549941046232873); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.002416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-09 01:56:48,217 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv2099156144:81 , srv912146513:82 , srv1153553428:83 , srv1319189526:83 , srv1334923670:83 , srv1444974627:83 , srv1890281929:83 , srv1062764260:84 , srv1354777676:84 , srv1430884532:84 , srv1444961641:84 , srv1766688890:84 , srv1774358642:84 , srv1819548513:84 , srv1932338355:84 , srv2122192450:84 , srv261292142:84 , srv422895581:84 , srv809833735:84 , srv86425939:84 , srv939171106:84 , srv1014734461:85 , srv109507858:85 , srv1103992068:85 , srv1117142075:85 , srv1122904371:85 , srv1129291300:85 , srv1131421187:85 , srv1149607492:85 , srv1218169184:85 , srv1388078833:85 , srv1397892632:85 , srv1458870020:85 , srv1466735939:85 , srv1470729187:85 , srv1559294963:85 , srv1675163861:85 , srv1698313941:85 , srv1706339786:85 , srv1786970959:85 , srv1846985547:85 , srv1855568756:85 , srv1937557087:85 , srv1938861902:85 , srv2043993912:85 , srv2076707974:85 , srv238641144:85 , srv305117158:85 , srv357010475:85 , srv42771606:85 , srv440425570:85 , srv491540662:85 , srv568869182:85 , srv606768633:85 , srv615946819:85 , srv656477577:85 , srv915404727:85 , srv930478406:85 , srv940045809:85 , srv1092333729:86 , srv1108382202:86 , srv1164938172:86 , srv1209040471:86 , srv1265713700:86 , srv1283850630:86 , srv146980876:86 , srv1471919441:86 , srv1540841235:86 , srv1695057258:86 , srv1733131751:86 , srv1822980460:86 , srv1861311921:86 , srv297524976:86 , srv327558961:86 , srv446446669:86 , srv498405556:86 , srv634112752:86 , srv680294968:86 , srv780335142:86 , srv825741765:86 , srv944552158:86 , srv1033261441:87 , srv116543514:87 , srv1191155661:87 , srv1551364951:87 , srv1584598837:87 , srv1860973999:87 , srv1961366229:87 , srv2117681265:87 , srv229904173:87 , srv23115946:87 , srv52967638:87 , srv570707944:87 , srv581240746:87 , srv699244241:87 , srv765281136:87 , srv971080517:87 , srv1112584644:88 , srv990896250:88 , srv1295779276:556 }
2023-12-09 01:56:48,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-09 01:56:48,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1706339786=42, srv1444974627=29, srv116543514=15, srv990896250=99, srv1153553428=13, srv1846985547=49, srv680294968=85, srv825741765=90, srv940045809=96, srv1218169184=18, srv1283850630=20, srv1584598837=38, srv1540841235=35, srv305117158=68, srv780335142=88, srv52967638=77, srv581240746=80, srv1695057258=40, srv912146513=92, srv1466735939=31, srv971080517=98, srv1822980460=48, srv238641144=65, srv915404727=93, srv297524976=67, srv1786970959=46, srv1733131751=43, srv357010475=70, srv446446669=74, srv606768633=81, srv1861311921=52, srv422895581=71, srv568869182=78, srv699244241=86, srv1112584644=7, srv930478406=94, srv491540662=75, srv42771606=72, srv1397892632=26, srv2043993912=58, srv440425570=73, srv634112752=83, srv1470729187=33, srv1471919441=34, srv1766688890=44, srv109507858=4, srv1062764260=2, srv23115946=64, srv1458870020=30, srv1265713700=19, srv1430884532=27, srv1774358642=45, srv1937557087=55, srv1129291300=10, srv1117142075=8, srv1961366229=57, srv1295779276=21, srv86425939=91, srv261292142=66, srv2099156144=60, srv1103992068=5, srv615946819=82, srv1209040471=17, srv1675163861=39, srv2122192450=62, srv1819548513=47, srv1938861902=56, srv765281136=87, srv1014734461=0, srv1319189526=22, srv2076707974=59, srv327558961=69, srv1164938172=14, srv1122904371=9, srv1033261441=1, srv656477577=84, srv570707944=79, srv1334923670=23, srv809833735=89, srv939171106=95, srv1559294963=37, srv1932338355=54, srv229904173=63, srv1855568756=50, srv1860973999=51, srv2117681265=61, srv1698313941=41, srv944552158=97, srv1108382202=6, srv1131421187=11, srv1551364951=36, srv1890281929=53, srv1149607492=12, srv1354777676=24, srv498405556=76, srv1191155661=16, srv1444961641=28, srv1092333729=3, srv146980876=32, srv1388078833=25} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-09 01:56:48,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-09 01:56:48,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-09 01:56:48,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-09 01:56:48,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-09 01:56:48,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-09 01:56:48,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-09 01:56:48,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-09 01:56:48,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-09 01:56:48,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-09 01:56:48,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-09 01:56:48,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-09 01:56:48,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-09 01:56:48,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-09 01:56:48,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-09 01:56:48,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-09 01:56:48,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-09 01:56:48,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-09 01:56:48,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-09 01:56:48,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-09 01:56:48,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-09 01:56:48,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-09 01:56:48,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-09 01:56:48,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-09 01:56:48,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-09 01:56:48,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-09 01:56:48,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-09 01:56:48,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-09 01:56:48,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-09 01:56:48,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-09 01:56:48,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-09 01:56:48,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-09 01:56:48,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-09 01:56:48,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-09 01:56:48,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-09 01:56:48,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-09 01:56:48,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-09 01:56:48,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-09 01:56:48,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-09 01:56:48,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-09 01:56:48,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-09 01:56:48,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-09 01:56:48,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-09 01:56:48,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-09 01:56:48,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-09 01:56:48,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-09 01:56:48,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-09 01:56:48,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-09 01:56:48,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-09 01:56:48,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-09 01:56:48,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-09 01:56:48,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-09 01:56:48,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-09 01:56:48,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-09 01:56:48,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-09 01:56:48,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-09 01:56:48,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-09 01:56:48,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-09 01:56:48,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-09 01:56:48,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-09 01:56:48,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-09 01:56:48,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-09 01:56:48,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-09 01:56:48,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-09 01:56:48,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-09 01:56:48,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-09 01:56:48,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-09 01:56:48,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-09 01:56:48,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-09 01:56:48,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-09 01:56:48,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-09 01:56:48,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-09 01:56:48,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-09 01:56:48,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-09 01:56:48,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-09 01:56:48,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-09 01:56:48,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-09 01:56:48,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-09 01:56:48,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-09 01:56:48,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-09 01:56:48,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-09 01:56:48,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-09 01:56:48,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-09 01:56:48,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-09 01:56:48,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-09 01:56:48,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-09 01:56:48,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-09 01:56:48,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-09 01:56:48,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-09 01:56:48,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-09 01:56:48,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-09 01:56:48,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-09 01:56:48,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-09 01:56:48,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-09 01:56:48,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-09 01:56:48,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-09 01:56:48,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-09 01:56:48,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-09 01:56:48,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-09 01:56:48,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-09 01:56:48,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-09 01:56:48,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-09 01:56:48,324 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005549941046232873); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005014009553935699); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005549941046232873); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.002416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-09 01:56:48,325 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-12-09 01:56:48,325 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=2.668679375871612E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005549941046232873); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005014009553935699); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005549941046232873); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.002416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-12-09 01:56:48,579 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 329 ms to try 5000 different iterations.  Found a solution that moves 190 regions; Going from a computed imbalance of 2.668679375871612E-4 to a new imbalance of 1.895567855048867E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.868392354571063E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.021111111111111112); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.868392354571063E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-09 01:56:48,581 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1706339786:83 , srv1860973999:83 , srv1938861902:83 , srv2043993912:83 , srv23115946:83 , srv939171106:83 , srv990896250:83 , srv1033261441:84 , srv1092333729:84 , srv109507858:84 , srv1112584644:84 , srv1129291300:84 , srv1149607492:84 , srv1153553428:84 , srv1164938172:84 , srv1191155661:84 , srv1265713700:84 , srv1319189526:84 , srv1354777676:84 , srv1388078833:84 , srv1430884532:84 , srv1466735939:84 , srv1540841235:84 , srv1551364951:84 , srv1559294963:84 , srv1584598837:84 , srv1675163861:84 , srv1766688890:84 , srv1822980460:84 , srv1861311921:84 , srv1932338355:84 , srv1961366229:84 , srv2076707974:84 , srv2099156144:84 , srv2117681265:84 , srv229904173:84 , srv297524976:84 , srv52967638:84 , srv581240746:84 , srv656477577:84 , srv699244241:84 , srv765281136:84 , srv912146513:84 , srv930478406:84 , srv940045809:84 , srv944552158:84 , srv971080517:84 , srv1062764260:85 , srv1108382202:85 , srv1117142075:85 , srv1122904371:85 , srv1131421187:85 , srv116543514:85 , srv1209040471:85 , srv1283850630:85 , srv1334923670:85 , srv1397892632:85 , srv1444961641:85 , srv1444974627:85 , srv1458870020:85 , srv146980876:85 , srv1470729187:85 , srv1471919441:85 , srv1695057258:85 , srv1698313941:85 , srv1733131751:85 , srv1774358642:85 , srv1786970959:85 , srv1819548513:85 , srv1846985547:85 , srv1855568756:85 , srv1890281929:85 , srv1937557087:85 , srv2122192450:85 , srv238641144:85 , srv261292142:85 , srv305117158:85 , srv327558961:85 , srv357010475:85 , srv422895581:85 , srv42771606:85 , srv440425570:85 , srv446446669:85 , srv491540662:85 , srv498405556:85 , srv568869182:85 , srv570707944:85 , srv606768633:85 , srv634112752:85 , srv680294968:85 , srv780335142:85 , srv809833735:85 , srv825741765:85 , srv86425939:85 , srv915404727:85 , srv1014734461:86 , srv1103992068:86 , srv1218169184:86 , srv615946819:86 , srv1295779276:635 }
2023-12-09 01:56:48,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-09 01:56:48,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1706339786=42, srv1444974627=29, srv116543514=15, srv990896250=99, srv1153553428=13, srv1846985547=49, srv680294968=85, srv825741765=90, srv940045809=96, srv1218169184=18, srv1283850630=20, srv1584598837=38, srv1540841235=35, srv305117158=68, srv780335142=88, srv52967638=77, srv581240746=80, srv1695057258=40, srv912146513=92, srv1466735939=31, srv971080517=98, srv1822980460=48, srv238641144=65, srv915404727=93, srv297524976=67, srv1786970959=46, srv1733131751=43, srv357010475=70, srv446446669=74, srv606768633=81, srv1861311921=52, srv422895581=71, srv568869182=78, srv699244241=86, srv1112584644=7, srv930478406=94, srv491540662=75, srv42771606=72, srv1397892632=26, srv2043993912=58, srv440425570=73, srv634112752=83, srv1470729187=33, srv1471919441=34, srv1766688890=44, srv109507858=4, srv1062764260=2, srv23115946=64, srv1458870020=30, srv1265713700=19, srv1430884532=27, srv1774358642=45, srv1937557087=55, srv1129291300=10, srv1117142075=8, srv1961366229=57, srv1295779276=21, srv86425939=91, srv261292142=66, srv2099156144=60, srv1103992068=5, srv615946819=82, srv1209040471=17, srv1675163861=39, srv2122192450=62, srv1819548513=47, srv1938861902=56, srv765281136=87, srv1014734461=0, srv1319189526=22, srv2076707974=59, srv327558961=69, srv1164938172=14, srv1122904371=9, srv1033261441=1, srv656477577=84, srv570707944=79, srv1334923670=23, srv809833735=89, srv939171106=95, srv1559294963=37, srv1932338355=54, srv229904173=63, srv1855568756=50, srv1860973999=51, srv2117681265=61, srv1698313941=41, srv944552158=97, srv1108382202=6, srv1131421187=11, srv1551364951=36, srv1890281929=53, srv1149607492=12, srv1354777676=24, srv498405556=76, srv1191155661=16, srv1444961641=28, srv1092333729=3, srv146980876=32, srv1388078833=25} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-09 01:56:48,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-09 01:56:48,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-09 01:56:48,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-09 01:56:48,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-09 01:56:48,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-09 01:56:48,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-09 01:56:48,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-09 01:56:48,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-09 01:56:48,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-09 01:56:48,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-09 01:56:48,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-09 01:56:48,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-09 01:56:48,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-09 01:56:48,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-09 01:56:48,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-09 01:56:48,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-09 01:56:48,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-09 01:56:48,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-09 01:56:48,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-09 01:56:48,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-09 01:56:48,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-09 01:56:48,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-09 01:56:48,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-09 01:56:48,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-09 01:56:48,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-09 01:56:48,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-09 01:56:48,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-09 01:56:48,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-09 01:56:48,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-09 01:56:48,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-09 01:56:48,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-09 01:56:48,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-09 01:56:48,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-09 01:56:48,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-09 01:56:48,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-09 01:56:48,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-09 01:56:48,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-09 01:56:48,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-09 01:56:48,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-09 01:56:48,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-09 01:56:48,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-09 01:56:48,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-09 01:56:48,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-09 01:56:48,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-09 01:56:48,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-09 01:56:48,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-09 01:56:48,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-09 01:56:48,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-09 01:56:48,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-09 01:56:48,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-09 01:56:48,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-09 01:56:48,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-09 01:56:48,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-09 01:56:48,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-09 01:56:48,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-09 01:56:48,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-09 01:56:48,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-09 01:56:48,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-09 01:56:48,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-09 01:56:48,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-09 01:56:48,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-09 01:56:48,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-09 01:56:48,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-09 01:56:48,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-09 01:56:48,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-09 01:56:48,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-09 01:56:48,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-09 01:56:48,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-09 01:56:48,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-09 01:56:48,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-09 01:56:48,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-09 01:56:48,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-09 01:56:48,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-09 01:56:48,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-09 01:56:48,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-09 01:56:48,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-09 01:56:48,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-09 01:56:48,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-09 01:56:48,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-09 01:56:48,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-09 01:56:48,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-09 01:56:48,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-09 01:56:48,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-09 01:56:48,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-09 01:56:48,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-09 01:56:48,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-09 01:56:48,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-09 01:56:48,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-09 01:56:48,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-09 01:56:48,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-09 01:56:48,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-09 01:56:48,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-09 01:56:48,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-09 01:56:48,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-09 01:56:48,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-09 01:56:48,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-09 01:56:48,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-09 01:56:48,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-09 01:56:48,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-09 01:56:48,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-09 01:56:48,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-09 01:56:48,744 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.7625270115099008E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.868392354571063E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.868392354571063E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-09 01:56:48,744 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1706339786:83 , srv1860973999:83 , srv1938861902:83 , srv2043993912:83 , srv23115946:83 , srv939171106:83 , srv990896250:83 , srv1033261441:84 , srv1092333729:84 , srv109507858:84 , srv1112584644:84 , srv1129291300:84 , srv1149607492:84 , srv1153553428:84 , srv1164938172:84 , srv1191155661:84 , srv1265713700:84 , srv1319189526:84 , srv1354777676:84 , srv1388078833:84 , srv1430884532:84 , srv1466735939:84 , srv1540841235:84 , srv1551364951:84 , srv1559294963:84 , srv1584598837:84 , srv1675163861:84 , srv1766688890:84 , srv1822980460:84 , srv1861311921:84 , srv1932338355:84 , srv1961366229:84 , srv2076707974:84 , srv2099156144:84 , srv2117681265:84 , srv229904173:84 , srv297524976:84 , srv52967638:84 , srv581240746:84 , srv656477577:84 , srv699244241:84 , srv765281136:84 , srv912146513:84 , srv930478406:84 , srv940045809:84 , srv944552158:84 , srv971080517:84 , srv1062764260:85 , srv1108382202:85 , srv1117142075:85 , srv1122904371:85 , srv1131421187:85 , srv116543514:85 , srv1209040471:85 , srv1283850630:85 , srv1334923670:85 , srv1397892632:85 , srv1444961641:85 , srv1444974627:85 , srv1458870020:85 , srv146980876:85 , srv1470729187:85 , srv1471919441:85 , srv1695057258:85 , srv1698313941:85 , srv1733131751:85 , srv1774358642:85 , srv1786970959:85 , srv1819548513:85 , srv1846985547:85 , srv1855568756:85 , srv1890281929:85 , srv1937557087:85 , srv2122192450:85 , srv238641144:85 , srv261292142:85 , srv305117158:85 , srv327558961:85 , srv357010475:85 , srv422895581:85 , srv42771606:85 , srv440425570:85 , srv446446669:85 , srv491540662:85 , srv498405556:85 , srv568869182:85 , srv570707944:85 , srv606768633:85 , srv634112752:85 , srv680294968:85 , srv780335142:85 , srv809833735:85 , srv825741765:85 , srv86425939:85 , srv915404727:85 , srv1014734461:86 , srv1103992068:86 , srv1218169184:86 , srv615946819:86 , srv1295779276:635 }
2023-12-09 01:56:48,798 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=302 (was 284) - SystemLoadAverage LEAK? -, ProcessCount=172 (was 172), AvailableMemoryMB=2373 (was 2856)
2023-12-09 01:56:48,818 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=292, MaxFileDescriptor=60000, SystemLoadAverage=302, ProcessCount=172, AvailableMemoryMB=2373
2023-12-09 01:56:48,819 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-12-09 01:56:48,819 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-12-09 01:56:48,837 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1052637400:84 , srv122971258:84 , srv1309255162:84 , srv1476118179:84 , srv1477070258:84 , srv1644539585:84 , srv1683609531:84 , srv1722590209:84 , srv1772678085:84 , srv1788843613:84 , srv1891781668:84 , srv1950959885:84 , srv202181705:84 , srv2089729496:84 , srv2105897058:84 , srv2140123463:84 , srv260086855:84 , srv384472387:84 , srv481820746:84 , srv526740239:84 , srv535099077:84 , srv564083199:84 , srv622957788:84 , srv679574863:84 , srv773537997:84 , srv810412438:84 , srv886562633:84 , srv950240880:84 , srv952943580:84 , srv1087878242:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-12-09 01:56:48,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-09 01:56:48,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2089729496=14, srv2140123463=16, srv1476118179=4, srv481820746=19, srv679574863=24, srv1722590209=8, srv122971258=2, srv773537997=25, srv1477070258=5, srv535099077=21, srv1309255162=3, srv1788843613=10, srv384472387=18, srv1891781668=11, srv260086855=17, srv564083199=22, srv1087878242=1, srv1052637400=0, srv526740239=20, srv810412438=26, srv1644539585=6, srv1772678085=9, srv886562633=27, srv1683609531=7, srv1950959885=12, srv622957788=23, srv950240880=28, srv202181705=13, srv2105897058=15, srv952943580=29} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-09 01:56:48,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-09 01:56:48,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-09 01:56:48,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-09 01:56:48,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-09 01:56:48,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-09 01:56:48,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-09 01:56:48,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-09 01:56:48,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-09 01:56:48,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-09 01:56:48,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-09 01:56:48,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-09 01:56:48,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-09 01:56:48,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-09 01:56:48,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-09 01:56:48,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-09 01:56:48,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-09 01:56:48,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-09 01:56:48,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-09 01:56:48,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-09 01:56:48,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-09 01:56:48,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-09 01:56:48,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-09 01:56:48,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-09 01:56:48,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-09 01:56:48,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-09 01:56:48,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-09 01:56:48,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-09 01:56:48,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-09 01:56:48,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-09 01:56:48,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-09 01:56:48,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-12-09 01:56:48,885 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1360173460855714, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-09 01:56:48,885 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912621629480419, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1360173460855714, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-12-09 01:58:48,864 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 9600382 different iterations.  Found a solution that moves 1838 regions; Going from a computed imbalance of 0.9912621629480419 to a new imbalance of 4.2899836916600065E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6807407407407408, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-09 01:58:48,869 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1052637400:90 , srv1087878242:90 , srv122971258:90 , srv1309255162:90 , srv1476118179:90 , srv1477070258:90 , srv1644539585:90 , srv1683609531:90 , srv1722590209:90 , srv1772678085:90 , srv1788843613:90 , srv1891781668:90 , srv1950959885:90 , srv202181705:90 , srv2089729496:90 , srv2105897058:90 , srv2140123463:90 , srv260086855:90 , srv384472387:90 , srv481820746:90 , srv526740239:90 , srv535099077:90 , srv564083199:90 , srv622957788:90 , srv679574863:90 , srv773537997:90 , srv810412438:90 , srv886562633:90 , srv950240880:90 , srv952943580:90 }
2023-12-09 01:58:48,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-09 01:58:48,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2089729496=14, srv2140123463=16, srv1476118179=4, srv481820746=19, srv679574863=24, srv1722590209=8, srv122971258=2, srv773537997=25, srv1477070258=5, srv535099077=21, srv1309255162=3, srv1788843613=10, srv384472387=18, srv1891781668=11, srv260086855=17, srv564083199=22, srv1087878242=1, srv1052637400=0, srv526740239=20, srv810412438=26, srv1644539585=6, srv1772678085=9, srv886562633=27, srv1683609531=7, srv1950959885=12, srv622957788=23, srv950240880=28, srv202181705=13, srv2105897058=15, srv952943580=29} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-09 01:58:48,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-09 01:58:48,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-09 01:58:48,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-09 01:58:48,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-09 01:58:48,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-09 01:58:48,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-09 01:58:48,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-09 01:58:48,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-09 01:58:48,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-09 01:58:48,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-09 01:58:48,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-09 01:58:48,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-09 01:58:48,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-09 01:58:48,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-09 01:58:48,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-09 01:58:48,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-09 01:58:48,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-09 01:58:48,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-09 01:58:48,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-09 01:58:48,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-09 01:58:48,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-09 01:58:48,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-09 01:58:48,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-09 01:58:48,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-09 01:58:48,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-09 01:58:48,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-09 01:58:48,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-09 01:58:48,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-09 01:58:48,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-09 01:58:48,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-09 01:58:48,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-12-09 01:58:48,883 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-09 01:58:48,883 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1052637400:90 , srv1087878242:90 , srv122971258:90 , srv1309255162:90 , srv1476118179:90 , srv1477070258:90 , srv1644539585:90 , srv1683609531:90 , srv1722590209:90 , srv1772678085:90 , srv1788843613:90 , srv1891781668:90 , srv1950959885:90 , srv202181705:90 , srv2089729496:90 , srv2105897058:90 , srv2140123463:90 , srv260086855:90 , srv384472387:90 , srv481820746:90 , srv526740239:90 , srv535099077:90 , srv564083199:90 , srv622957788:90 , srv679574863:90 , srv773537997:90 , srv810412438:90 , srv886562633:90 , srv950240880:90 , srv952943580:90 }
2023-12-09 01:58:48,894 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 292), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=177 (was 302), ProcessCount=168 (was 172), AvailableMemoryMB=3441 (was 2373) - AvailableMemoryMB LEAK? -