2023-11-24 02:53:16,505 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-11-24 02:53:16,842 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-11-24 02:53:16,961 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-11-24 02:53:17,031 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-24 02:53:17,039 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-11-24 02:53:17,052 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=321, ProcessCount=170, AvailableMemoryMB=8108
2023-11-24 02:53:17,054 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-24 02:53:17,054 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-11-24 02:53:17,171 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1011335088:84 , srv1023258354:84 , srv1030237206:84 , srv1047537726:84 , srv1072090062:84 , srv112358043:84 , srv1149377527:84 , srv1154613002:84 , srv1162684455:84 , srv1165432180:84 , srv1176476804:84 , srv1192008256:84 , srv11950687:84 , srv1216448736:84 , srv1217420278:84 , srv1235115868:84 , srv124158037:84 , srv1256012365:84 , srv1260065762:84 , srv1292387647:84 , srv1293742673:84 , srv1297526899:84 , srv130569168:84 , srv1319696283:84 , srv1354360676:84 , srv1370562333:84 , srv1387580634:84 , srv1417434795:84 , srv1456738428:84 , srv1480275521:84 , srv1496212035:84 , srv1515118140:84 , srv1516060451:84 , srv1531307561:84 , srv1570627099:84 , srv1582266676:84 , srv1583063226:84 , srv1589494520:84 , srv1608303718:84 , srv1613119843:84 , srv1627214712:84 , srv1676510302:84 , srv1708255259:84 , srv1744225492:84 , srv1763967513:84 , srv1774097822:84 , srv1776098336:84 , srv1803674097:84 , srv1863877801:84 , srv1869856496:84 , srv1872599396:84 , srv1931432803:84 , srv193600327:84 , srv1957015376:84 , srv1976379829:84 , srv1976637450:84 , srv1979763363:84 , srv1983557005:84 , srv2010153321:84 , srv2017293954:84 , srv2039409182:84 , srv2054911381:84 , srv2080486709:84 , srv2102014795:84 , srv2132233200:84 , srv2133666664:84 , srv2146419750:84 , srv220117556:84 , srv237827580:84 , srv240754922:84 , srv345037907:84 , srv367697521:84 , srv379015622:84 , srv41152192:84 , srv458870246:84 , srv489930257:84 , srv503520151:84 , srv504257811:84 , srv509380634:84 , srv519764500:84 , srv549751938:84 , srv590134587:84 , srv597380146:84 , srv624286773:84 , srv640932582:84 , srv704955788:84 , srv71818017:84 , srv73485378:84 , srv76779757:84 , srv776965031:84 , srv804007425:84 , srv820424968:84 , srv829830915:84 , srv840247320:84 , srv848886138:84 , srv849183247:84 , srv906086149:84 , srv948514491:84 , srv994337915:84 , srv669081196:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-11-24 02:53:17,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-24 02:53:17,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1023258354=1, srv1979763363=56, srv2146419750=66, srv1260065762=18, srv1869856496=49, srv1863877801=48, srv458870246=74, srv1456738428=28, srv519764500=79, srv704955788=86, srv1776098336=46, srv1165432180=9, srv1983557005=57, srv41152192=73, srv1515118140=31, srv804007425=91, srv71818017=87, srv669081196=85, srv1292387647=19, srv1708255259=42, srv2132233200=64, srv590134587=81, srv1217420278=14, srv640932582=84, srv1589494520=37, srv1803674097=47, srv1931432803=51, srv1872599396=50, srv1235115868=15, srv1387580634=26, srv848886138=95, srv1583063226=36, srv840247320=94, srv1030237206=2, srv503520151=76, srv1417434795=27, srv237827580=68, srv820424968=92, srv240754922=69, srv1516060451=32, srv504257811=77, srv76779757=89, srv1531307561=33, srv2010153321=58, srv1176476804=10, srv509380634=78, srv994337915=99, srv73485378=88, srv2054911381=61, srv1192008256=11, srv1582266676=35, srv367697521=71, srv489930257=75, srv1047537726=3, srv1957015376=53, srv1774097822=45, srv1354360676=24, srv193600327=52, srv345037907=70, srv1149377527=6, srv130569168=22, srv597380146=82, srv220117556=67, srv2080486709=62, srv1162684455=8, srv124158037=16, srv1319696283=23, srv624286773=83, srv906086149=97, srv1763967513=44, srv1496212035=30, srv829830915=93, srv1613119843=39, srv1480275521=29, srv2039409182=60, srv1744225492=43, srv1297526899=21, srv549751938=80, srv2017293954=59, srv948514491=98, srv1154613002=7, srv1570627099=34, srv1976379829=54, srv2133666664=65, srv11950687=12, srv1256012365=17, srv1676510302=41, srv379015622=72, srv1976637450=55, srv112358043=5, srv1627214712=40, srv1370562333=25, srv849183247=96, srv1011335088=0, srv2102014795=63, srv776965031=90, srv1293742673=20, srv1216448736=13, srv1608303718=38, srv1072090062=4} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-24 02:53:17,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-24 02:53:17,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-24 02:53:17,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-24 02:53:17,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-24 02:53:17,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-24 02:53:17,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-24 02:53:17,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-24 02:53:17,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-24 02:53:17,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-24 02:53:17,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-24 02:53:17,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-24 02:53:17,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-24 02:53:17,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-24 02:53:17,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-24 02:53:17,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-24 02:53:17,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-24 02:53:17,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-24 02:53:17,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-24 02:53:17,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-24 02:53:17,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-24 02:53:17,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-24 02:53:17,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-24 02:53:17,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-24 02:53:17,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-24 02:53:17,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-24 02:53:17,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-24 02:53:17,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-24 02:53:17,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-24 02:53:17,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-24 02:53:17,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-24 02:53:17,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-24 02:53:17,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-24 02:53:17,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-24 02:53:17,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-24 02:53:17,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-24 02:53:17,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-24 02:53:17,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-24 02:53:17,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-24 02:53:17,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-24 02:53:17,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-24 02:53:17,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-24 02:53:17,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-24 02:53:17,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-24 02:53:17,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-24 02:53:17,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-24 02:53:17,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-24 02:53:17,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-24 02:53:17,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-24 02:53:17,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-24 02:53:17,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-24 02:53:17,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-24 02:53:17,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-24 02:53:17,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-24 02:53:17,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-24 02:53:17,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-24 02:53:17,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-24 02:53:17,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-24 02:53:17,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-24 02:53:17,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-24 02:53:17,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-24 02:53:17,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-24 02:53:17,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-24 02:53:17,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-24 02:53:17,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-24 02:53:17,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-24 02:53:17,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-24 02:53:17,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-24 02:53:17,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-24 02:53:17,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-24 02:53:17,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-24 02:53:17,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-24 02:53:17,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-24 02:53:17,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-24 02:53:17,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-24 02:53:17,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-24 02:53:17,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-24 02:53:17,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-24 02:53:17,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-24 02:53:17,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-24 02:53:17,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-24 02:53:17,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-24 02:53:17,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-24 02:53:17,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-24 02:53:17,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-24 02:53:17,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-24 02:53:17,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-24 02:53:17,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-24 02:53:17,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-24 02:53:17,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-24 02:53:17,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-24 02:53:17,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-24 02:53:17,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-24 02:53:17,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-24 02:53:17,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-24 02:53:17,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-24 02:53:17,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-24 02:53:17,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-24 02:53:17,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-24 02:53:17,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-24 02:53:17,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-24 02:53:17,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-24 02:53:17,448 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-24 02:53:17,448 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-24 02:53:17,449 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-24 02:53:17,850 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 549 ms to try 5000 different iterations.  Found a solution that moves 4688 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18009925091272247. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040829716125850915, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05878874656584702, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5208888888888888, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040829716125850915, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17525, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2425, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-24 02:53:17,866 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1387580634:81 , srv1292387647:82 , srv1763967513:82 , srv504257811:82 , srv549751938:82 , srv849183247:82 , srv1570627099:83 , srv237827580:83 , srv804007425:83 , srv1011335088:84 , srv11950687:84 , srv1319696283:84 , srv1480275521:84 , srv1582266676:84 , srv1744225492:84 , srv1983557005:84 , srv2039409182:84 , srv2080486709:84 , srv2102014795:84 , srv458870246:84 , srv1149377527:85 , srv1176476804:85 , srv1217420278:85 , srv124158037:85 , srv1256012365:85 , srv1516060451:85 , srv1589494520:85 , srv1931432803:85 , srv193600327:85 , srv1979763363:85 , srv2017293954:85 , srv220117556:85 , srv367697521:85 , srv489930257:85 , srv590134587:85 , srv597380146:85 , srv71818017:85 , srv840247320:85 , srv1023258354:86 , srv1030237206:86 , srv1072090062:86 , srv1162684455:86 , srv1216448736:86 , srv1260065762:86 , srv1456738428:86 , srv1496212035:86 , srv1776098336:86 , srv1803674097:86 , srv1872599396:86 , srv1976379829:86 , srv2010153321:86 , srv345037907:86 , srv503520151:86 , srv519764500:86 , srv776965031:86 , srv820424968:86 , srv848886138:86 , srv994337915:86 , srv1154613002:87 , srv1192008256:87 , srv1354360676:87 , srv1417434795:87 , srv1708255259:87 , srv1774097822:87 , srv1863877801:87 , srv1957015376:87 , srv2133666664:87 , srv240754922:87 , srv73485378:87 , srv948514491:87 , srv1047537726:88 , srv1293742673:88 , srv1297526899:88 , srv1370562333:88 , srv1515118140:88 , srv1608303718:88 , srv1869856496:88 , srv2132233200:88 , srv2146419750:88 , srv379015622:88 , srv41152192:88 , srv624286773:88 , srv704955788:88 , srv829830915:88 , srv112358043:89 , srv1976637450:89 , srv640932582:89 , srv906086149:89 , srv130569168:90 , srv1531307561:90 , srv1583063226:90 , srv1627214712:90 , srv1676510302:90 , srv2054911381:90 , srv1165432180:91 , srv76779757:91 , srv509380634:92 , srv1613119843:93 , srv1235115868:94 , srv669081196:453 }
2023-11-24 02:53:17,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-24 02:53:17,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1023258354=1, srv1979763363=56, srv2146419750=66, srv1260065762=18, srv1869856496=49, srv1863877801=48, srv458870246=74, srv1456738428=28, srv519764500=79, srv704955788=86, srv1776098336=46, srv1165432180=9, srv1983557005=57, srv41152192=73, srv1515118140=31, srv804007425=91, srv71818017=87, srv669081196=85, srv1292387647=19, srv1708255259=42, srv2132233200=64, srv590134587=81, srv1217420278=14, srv640932582=84, srv1589494520=37, srv1803674097=47, srv1931432803=51, srv1872599396=50, srv1235115868=15, srv1387580634=26, srv848886138=95, srv1583063226=36, srv840247320=94, srv1030237206=2, srv503520151=76, srv1417434795=27, srv237827580=68, srv820424968=92, srv240754922=69, srv1516060451=32, srv504257811=77, srv76779757=89, srv1531307561=33, srv2010153321=58, srv1176476804=10, srv509380634=78, srv994337915=99, srv73485378=88, srv2054911381=61, srv1192008256=11, srv1582266676=35, srv367697521=71, srv489930257=75, srv1047537726=3, srv1957015376=53, srv1774097822=45, srv1354360676=24, srv193600327=52, srv345037907=70, srv1149377527=6, srv130569168=22, srv597380146=82, srv220117556=67, srv2080486709=62, srv1162684455=8, srv124158037=16, srv1319696283=23, srv624286773=83, srv906086149=97, srv1763967513=44, srv1496212035=30, srv829830915=93, srv1613119843=39, srv1480275521=29, srv2039409182=60, srv1744225492=43, srv1297526899=21, srv549751938=80, srv2017293954=59, srv948514491=98, srv1154613002=7, srv1570627099=34, srv1976379829=54, srv2133666664=65, srv11950687=12, srv1256012365=17, srv1676510302=41, srv379015622=72, srv1976637450=55, srv112358043=5, srv1627214712=40, srv1370562333=25, srv849183247=96, srv1011335088=0, srv2102014795=63, srv776965031=90, srv1293742673=20, srv1216448736=13, srv1608303718=38, srv1072090062=4} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-24 02:53:17,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-24 02:53:17,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-24 02:53:17,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-24 02:53:17,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-24 02:53:17,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-24 02:53:17,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-24 02:53:17,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-24 02:53:17,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-24 02:53:17,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-24 02:53:17,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-24 02:53:17,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-24 02:53:17,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-24 02:53:17,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-24 02:53:17,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-24 02:53:17,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-24 02:53:17,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-24 02:53:17,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-24 02:53:17,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-24 02:53:17,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-24 02:53:17,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-24 02:53:17,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-24 02:53:17,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-24 02:53:17,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-24 02:53:17,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-24 02:53:17,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-24 02:53:17,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-24 02:53:17,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-24 02:53:17,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-24 02:53:17,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-24 02:53:17,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-24 02:53:17,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-24 02:53:17,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-24 02:53:17,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-24 02:53:17,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-24 02:53:17,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-24 02:53:17,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-24 02:53:17,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-24 02:53:17,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-24 02:53:17,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-24 02:53:17,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-24 02:53:17,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-24 02:53:17,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-24 02:53:17,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-24 02:53:17,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-24 02:53:17,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-24 02:53:17,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-24 02:53:17,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-24 02:53:17,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-24 02:53:17,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-24 02:53:17,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-24 02:53:17,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-24 02:53:17,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-24 02:53:17,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-24 02:53:17,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-24 02:53:17,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-24 02:53:17,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-24 02:53:17,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-24 02:53:17,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-24 02:53:17,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-24 02:53:17,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-24 02:53:17,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-24 02:53:17,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-24 02:53:17,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-24 02:53:17,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-24 02:53:17,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-24 02:53:17,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-24 02:53:17,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-24 02:53:17,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-24 02:53:17,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-24 02:53:17,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-24 02:53:17,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-24 02:53:17,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-24 02:53:17,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-24 02:53:17,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-24 02:53:17,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-24 02:53:17,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-24 02:53:17,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-24 02:53:17,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-24 02:53:17,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-24 02:53:17,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-24 02:53:17,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-24 02:53:17,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-24 02:53:17,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-24 02:53:17,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-24 02:53:17,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-24 02:53:17,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-24 02:53:17,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-24 02:53:17,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-24 02:53:17,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-24 02:53:17,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-24 02:53:17,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-24 02:53:17,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-24 02:53:17,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-24 02:53:17,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-24 02:53:17,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-24 02:53:17,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-24 02:53:17,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-24 02:53:17,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-24 02:53:17,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-24 02:53:17,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-24 02:53:17,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-24 02:53:18,062 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040829716125850915, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05878874656584702, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040829716125850915, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17525, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2425, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-24 02:53:18,062 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-24 02:53:18,062 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18006642483511667, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040829716125850915, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05878874656584702, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040829716125850915, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17525, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2425, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-24 02:53:18,342 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 414 ms to try 5000 different iterations.  Found a solution that moves 2648 regions; Going from a computed imbalance of 0.18006642483511667 to a new imbalance of 0.015423641021931828. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021616318952194954); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.032508308529617315, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2942222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021616318952194954); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012333333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.045, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-24 02:53:18,354 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1235115868:77 , srv1165432180:79 , srv1531307561:80 , srv1613119843:81 , srv2054911381:81 , srv76779757:81 , srv1627214712:82 , srv704955788:82 , srv906086149:82 , srv1047537726:83 , srv130569168:83 , srv1354360676:83 , srv1417434795:83 , srv1583063226:83 , srv1676510302:83 , srv2133666664:83 , srv112358043:84 , srv1192008256:84 , srv1293742673:84 , srv1297526899:84 , srv1370562333:84 , srv1515118140:84 , srv1774097822:84 , srv240754922:84 , srv41152192:84 , srv509380634:84 , srv640932582:84 , srv1023258354:85 , srv1030237206:85 , srv1072090062:85 , srv1216448736:85 , srv1217420278:85 , srv1456738428:85 , srv1589494520:85 , srv1931432803:85 , srv1976379829:85 , srv1976637450:85 , srv1983557005:85 , srv2146419750:85 , srv73485378:85 , srv776965031:85 , srv948514491:85 , srv1149377527:86 , srv1154613002:86 , srv1162684455:86 , srv1319696283:86 , srv1480275521:86 , srv1608303718:86 , srv1744225492:86 , srv1803674097:86 , srv1869856496:86 , srv1979763363:86 , srv2010153321:86 , srv2017293954:86 , srv2132233200:86 , srv367697521:86 , srv519764500:86 , srv71818017:86 , srv829830915:86 , srv848886138:86 , srv1176476804:87 , srv11950687:87 , srv124158037:87 , srv1256012365:87 , srv1496212035:87 , srv1776098336:87 , srv1872599396:87 , srv193600327:87 , srv1957015376:87 , srv2039409182:87 , srv2080486709:87 , srv220117556:87 , srv379015622:87 , srv590134587:87 , srv597380146:87 , srv624286773:87 , srv820424968:87 , srv840247320:87 , srv1011335088:88 , srv1260065762:88 , srv1516060451:88 , srv1582266676:88 , srv1763967513:88 , srv345037907:88 , srv804007425:88 , srv1292387647:89 , srv1570627099:89 , srv1708255259:89 , srv1863877801:89 , srv2102014795:89 , srv458870246:89 , srv503520151:89 , srv549751938:89 , srv849183247:89 , srv994337915:89 , srv489930257:90 , srv504257811:90 , srv1387580634:91 , srv237827580:91 , srv669081196:513 }
2023-11-24 02:53:18,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-24 02:53:18,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1023258354=1, srv1979763363=56, srv2146419750=66, srv1260065762=18, srv1869856496=49, srv1863877801=48, srv458870246=74, srv1456738428=28, srv519764500=79, srv704955788=86, srv1776098336=46, srv1165432180=9, srv1983557005=57, srv41152192=73, srv1515118140=31, srv804007425=91, srv71818017=87, srv669081196=85, srv1292387647=19, srv1708255259=42, srv2132233200=64, srv590134587=81, srv1217420278=14, srv640932582=84, srv1589494520=37, srv1803674097=47, srv1931432803=51, srv1872599396=50, srv1235115868=15, srv1387580634=26, srv848886138=95, srv1583063226=36, srv840247320=94, srv1030237206=2, srv503520151=76, srv1417434795=27, srv237827580=68, srv820424968=92, srv240754922=69, srv1516060451=32, srv504257811=77, srv76779757=89, srv1531307561=33, srv2010153321=58, srv1176476804=10, srv509380634=78, srv994337915=99, srv73485378=88, srv2054911381=61, srv1192008256=11, srv1582266676=35, srv367697521=71, srv489930257=75, srv1047537726=3, srv1957015376=53, srv1774097822=45, srv1354360676=24, srv193600327=52, srv345037907=70, srv1149377527=6, srv130569168=22, srv597380146=82, srv220117556=67, srv2080486709=62, srv1162684455=8, srv124158037=16, srv1319696283=23, srv624286773=83, srv906086149=97, srv1763967513=44, srv1496212035=30, srv829830915=93, srv1613119843=39, srv1480275521=29, srv2039409182=60, srv1744225492=43, srv1297526899=21, srv549751938=80, srv2017293954=59, srv948514491=98, srv1154613002=7, srv1570627099=34, srv1976379829=54, srv2133666664=65, srv11950687=12, srv1256012365=17, srv1676510302=41, srv379015622=72, srv1976637450=55, srv112358043=5, srv1627214712=40, srv1370562333=25, srv849183247=96, srv1011335088=0, srv2102014795=63, srv776965031=90, srv1293742673=20, srv1216448736=13, srv1608303718=38, srv1072090062=4} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-24 02:53:18,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-24 02:53:18,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-24 02:53:18,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-24 02:53:18,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-24 02:53:18,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-24 02:53:18,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-24 02:53:18,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-24 02:53:18,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-24 02:53:18,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-24 02:53:18,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-24 02:53:18,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-24 02:53:18,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-24 02:53:18,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-24 02:53:18,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-24 02:53:18,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-24 02:53:18,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-24 02:53:18,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-24 02:53:18,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-24 02:53:18,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-24 02:53:18,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-24 02:53:18,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-24 02:53:18,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-24 02:53:18,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-24 02:53:18,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-24 02:53:18,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-24 02:53:18,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-24 02:53:18,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-24 02:53:18,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-24 02:53:18,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-24 02:53:18,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-24 02:53:18,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-24 02:53:18,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-24 02:53:18,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-24 02:53:18,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-24 02:53:18,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-24 02:53:18,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-24 02:53:18,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-24 02:53:18,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-24 02:53:18,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-24 02:53:18,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-24 02:53:18,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-24 02:53:18,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-24 02:53:18,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-24 02:53:18,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-24 02:53:18,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-24 02:53:18,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-24 02:53:18,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-24 02:53:18,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-24 02:53:18,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-24 02:53:18,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-24 02:53:18,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-24 02:53:18,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-24 02:53:18,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-24 02:53:18,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-24 02:53:18,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-24 02:53:18,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-24 02:53:18,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-24 02:53:18,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-24 02:53:18,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-24 02:53:18,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-24 02:53:18,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-24 02:53:18,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-24 02:53:18,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-24 02:53:18,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-24 02:53:18,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-24 02:53:18,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-24 02:53:18,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-24 02:53:18,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-24 02:53:18,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-24 02:53:18,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-24 02:53:18,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-24 02:53:18,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-24 02:53:18,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-24 02:53:18,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-24 02:53:18,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-24 02:53:18,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-24 02:53:18,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-24 02:53:18,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-24 02:53:18,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-24 02:53:18,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-24 02:53:18,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-24 02:53:18,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-24 02:53:18,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-24 02:53:18,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-24 02:53:18,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-24 02:53:18,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-24 02:53:18,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-24 02:53:18,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-24 02:53:18,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-24 02:53:18,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-24 02:53:18,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-24 02:53:18,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-24 02:53:18,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-24 02:53:18,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-24 02:53:18,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-24 02:53:18,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-24 02:53:18,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-24 02:53:18,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-24 02:53:18,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-24 02:53:18,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-24 02:53:18,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-24 02:53:18,555 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021616318952194954); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.032508308529617315, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021616318952194954); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012333333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.045, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-24 02:53:18,555 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-24 02:53:18,555 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.015405099329632292, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021616318952194954); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.032508308529617315, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021616318952194954); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012333333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.045, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-24 02:53:18,825 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 403 ms to try 5000 different iterations.  Found a solution that moves 962 regions; Going from a computed imbalance of 0.015405099329632292 to a new imbalance of 0.0011994221139074665. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005534190117523058); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0057050590894718945); MoveCostFunction : (multiplier=7.0, imbalance=0.1068888888888889, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005534190117523058); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=7.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.005166666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-24 02:53:18,826 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1708255259:82 , srv1863877801:82 , srv379015622:82 , srv489930257:82 , srv503520151:82 , srv1869856496:83 , srv1957015376:83 , srv1976637450:83 , srv2132233200:83 , srv237827580:83 , srv509380634:83 , srv624286773:83 , srv829830915:83 , srv994337915:83 , srv112358043:84 , srv1154613002:84 , srv1260065762:84 , srv1608303718:84 , srv1613119843:84 , srv1776098336:84 , srv2146419750:84 , srv345037907:84 , srv640932582:84 , srv820424968:84 , srv1011335088:85 , srv1176476804:85 , srv124158037:85 , srv1293742673:85 , srv1297526899:85 , srv130569168:85 , srv1370562333:85 , srv1387580634:85 , srv1496212035:85 , srv1516060451:85 , srv1583063226:85 , srv1676510302:85 , srv1803674097:85 , srv1872599396:85 , srv2102014795:85 , srv458870246:85 , srv590134587:85 , srv76779757:85 , srv948514491:85 , srv1023258354:86 , srv1030237206:86 , srv1047537726:86 , srv1072090062:86 , srv1149377527:86 , srv1162684455:86 , srv1192008256:86 , srv11950687:86 , srv1256012365:86 , srv1292387647:86 , srv1456738428:86 , srv1515118140:86 , srv1570627099:86 , srv1582266676:86 , srv1627214712:86 , srv1774097822:86 , srv2039409182:86 , srv220117556:86 , srv367697521:86 , srv41152192:86 , srv504257811:86 , srv519764500:86 , srv549751938:86 , srv597380146:86 , srv71818017:86 , srv73485378:86 , srv776965031:86 , srv840247320:86 , srv848886138:86 , srv849183247:86 , srv906086149:86 , srv1165432180:87 , srv1216448736:87 , srv1217420278:87 , srv1235115868:87 , srv1417434795:87 , srv1480275521:87 , srv1531307561:87 , srv1589494520:87 , srv1744225492:87 , srv1763967513:87 , srv193600327:87 , srv1976379829:87 , srv1979763363:87 , srv2010153321:87 , srv2017293954:87 , srv2054911381:87 , srv2080486709:87 , srv240754922:87 , srv704955788:87 , srv804007425:87 , srv1319696283:88 , srv1354360676:88 , srv1931432803:88 , srv2133666664:88 , srv1983557005:89 , srv669081196:541 }
2023-11-24 02:53:18,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-24 02:53:18,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1023258354=1, srv1979763363=56, srv2146419750=66, srv1260065762=18, srv1869856496=49, srv1863877801=48, srv458870246=74, srv1456738428=28, srv519764500=79, srv704955788=86, srv1776098336=46, srv1165432180=9, srv1983557005=57, srv41152192=73, srv1515118140=31, srv804007425=91, srv71818017=87, srv669081196=85, srv1292387647=19, srv1708255259=42, srv2132233200=64, srv590134587=81, srv1217420278=14, srv640932582=84, srv1589494520=37, srv1803674097=47, srv1931432803=51, srv1872599396=50, srv1235115868=15, srv1387580634=26, srv848886138=95, srv1583063226=36, srv840247320=94, srv1030237206=2, srv503520151=76, srv1417434795=27, srv237827580=68, srv820424968=92, srv240754922=69, srv1516060451=32, srv504257811=77, srv76779757=89, srv1531307561=33, srv2010153321=58, srv1176476804=10, srv509380634=78, srv994337915=99, srv73485378=88, srv2054911381=61, srv1192008256=11, srv1582266676=35, srv367697521=71, srv489930257=75, srv1047537726=3, srv1957015376=53, srv1774097822=45, srv1354360676=24, srv193600327=52, srv345037907=70, srv1149377527=6, srv130569168=22, srv597380146=82, srv220117556=67, srv2080486709=62, srv1162684455=8, srv124158037=16, srv1319696283=23, srv624286773=83, srv906086149=97, srv1763967513=44, srv1496212035=30, srv829830915=93, srv1613119843=39, srv1480275521=29, srv2039409182=60, srv1744225492=43, srv1297526899=21, srv549751938=80, srv2017293954=59, srv948514491=98, srv1154613002=7, srv1570627099=34, srv1976379829=54, srv2133666664=65, srv11950687=12, srv1256012365=17, srv1676510302=41, srv379015622=72, srv1976637450=55, srv112358043=5, srv1627214712=40, srv1370562333=25, srv849183247=96, srv1011335088=0, srv2102014795=63, srv776965031=90, srv1293742673=20, srv1216448736=13, srv1608303718=38, srv1072090062=4} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-24 02:53:18,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-24 02:53:18,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-24 02:53:18,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-24 02:53:18,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-24 02:53:18,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-24 02:53:18,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-24 02:53:18,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-24 02:53:18,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-24 02:53:18,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-24 02:53:18,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-24 02:53:18,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-24 02:53:18,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-24 02:53:18,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-24 02:53:18,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-24 02:53:18,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-24 02:53:18,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-24 02:53:18,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-24 02:53:18,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-24 02:53:18,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-24 02:53:18,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-24 02:53:18,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-24 02:53:18,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-24 02:53:18,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-24 02:53:18,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-24 02:53:18,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-24 02:53:18,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-24 02:53:18,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-24 02:53:18,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-24 02:53:18,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-24 02:53:18,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-24 02:53:18,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-24 02:53:18,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-24 02:53:18,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-24 02:53:18,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-24 02:53:18,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-24 02:53:18,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-24 02:53:18,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-24 02:53:18,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-24 02:53:18,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-24 02:53:18,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-24 02:53:18,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-24 02:53:18,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-24 02:53:18,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-24 02:53:18,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-24 02:53:18,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-24 02:53:18,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-24 02:53:18,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-24 02:53:18,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-24 02:53:18,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-24 02:53:18,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-24 02:53:18,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-24 02:53:18,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-24 02:53:18,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-24 02:53:18,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-24 02:53:18,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-24 02:53:18,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-24 02:53:18,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-24 02:53:18,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-24 02:53:18,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-24 02:53:18,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-24 02:53:18,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-24 02:53:18,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-24 02:53:18,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-24 02:53:18,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-24 02:53:18,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-24 02:53:18,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-24 02:53:18,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-24 02:53:18,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-24 02:53:18,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-24 02:53:18,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-24 02:53:18,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-24 02:53:18,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-24 02:53:18,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-24 02:53:18,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-24 02:53:18,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-24 02:53:18,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-24 02:53:18,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-24 02:53:18,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-24 02:53:18,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-24 02:53:18,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-24 02:53:18,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-24 02:53:18,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-24 02:53:18,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-24 02:53:18,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-24 02:53:18,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-24 02:53:18,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-24 02:53:18,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-24 02:53:18,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-24 02:53:18,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-24 02:53:18,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-24 02:53:18,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-24 02:53:18,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-24 02:53:18,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-24 02:53:18,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-24 02:53:18,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-24 02:53:18,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-24 02:53:18,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-24 02:53:18,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-24 02:53:18,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-24 02:53:18,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-24 02:53:18,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-24 02:53:18,979 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005534190117523058); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0057050590894718945); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005534190117523058); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=7.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.005166666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-24 02:53:18,979 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-24 02:53:18,979 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.001192686045934599, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005534190117523058); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0057050590894718945); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005534190117523058); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=7.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.005166666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-24 02:53:19,390 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 526 ms to try 5000 different iterations.  Found a solution that moves 255 regions; Going from a computed imbalance of 0.001192686045934599 to a new imbalance of 2.1667179911798534E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0022721660362024225); MoveCostFunction : (multiplier=7.0, imbalance=0.028333333333333332, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-24 02:53:19,391 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1676510302:83 , srv193600327:83 , srv1983557005:83 , srv2010153321:83 , srv2133666664:83 , srv509380634:83 , srv1162684455:84 , srv1165432180:84 , srv1216448736:84 , srv1256012365:84 , srv1260065762:84 , srv1297526899:84 , srv130569168:84 , srv1319696283:84 , srv1354360676:84 , srv1480275521:84 , srv1496212035:84 , srv1516060451:84 , srv1570627099:84 , srv1582266676:84 , srv1583063226:84 , srv1613119843:84 , srv1627214712:84 , srv1744225492:84 , srv1863877801:84 , srv1872599396:84 , srv1931432803:84 , srv1976379829:84 , srv1979763363:84 , srv2017293954:84 , srv2054911381:84 , srv2080486709:84 , srv2102014795:84 , srv220117556:84 , srv240754922:84 , srv345037907:84 , srv41152192:84 , srv458870246:84 , srv504257811:84 , srv519764500:84 , srv590134587:84 , srv597380146:84 , srv73485378:84 , srv804007425:84 , srv840247320:84 , srv1011335088:85 , srv1023258354:85 , srv1030237206:85 , srv1047537726:85 , srv1072090062:85 , srv112358043:85 , srv1149377527:85 , srv1154613002:85 , srv1176476804:85 , srv1192008256:85 , srv11950687:85 , srv1217420278:85 , srv1235115868:85 , srv1292387647:85 , srv1293742673:85 , srv1370562333:85 , srv1387580634:85 , srv1417434795:85 , srv1456738428:85 , srv1515118140:85 , srv1531307561:85 , srv1589494520:85 , srv1608303718:85 , srv1708255259:85 , srv1763967513:85 , srv1774097822:85 , srv1776098336:85 , srv1803674097:85 , srv1869856496:85 , srv1957015376:85 , srv1976637450:85 , srv2039409182:85 , srv2132233200:85 , srv2146419750:85 , srv237827580:85 , srv367697521:85 , srv379015622:85 , srv489930257:85 , srv503520151:85 , srv549751938:85 , srv624286773:85 , srv640932582:85 , srv704955788:85 , srv71818017:85 , srv76779757:85 , srv776965031:85 , srv820424968:85 , srv829830915:85 , srv848886138:85 , srv849183247:85 , srv906086149:85 , srv948514491:85 , srv994337915:85 , srv124158037:86 , srv669081196:635 }
2023-11-24 02:53:19,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-24 02:53:19,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1023258354=1, srv1979763363=56, srv2146419750=66, srv1260065762=18, srv1869856496=49, srv1863877801=48, srv458870246=74, srv1456738428=28, srv519764500=79, srv704955788=86, srv1776098336=46, srv1165432180=9, srv1983557005=57, srv41152192=73, srv1515118140=31, srv804007425=91, srv71818017=87, srv669081196=85, srv1292387647=19, srv1708255259=42, srv2132233200=64, srv590134587=81, srv1217420278=14, srv640932582=84, srv1589494520=37, srv1803674097=47, srv1931432803=51, srv1872599396=50, srv1235115868=15, srv1387580634=26, srv848886138=95, srv1583063226=36, srv840247320=94, srv1030237206=2, srv503520151=76, srv1417434795=27, srv237827580=68, srv820424968=92, srv240754922=69, srv1516060451=32, srv504257811=77, srv76779757=89, srv1531307561=33, srv2010153321=58, srv1176476804=10, srv509380634=78, srv994337915=99, srv73485378=88, srv2054911381=61, srv1192008256=11, srv1582266676=35, srv367697521=71, srv489930257=75, srv1047537726=3, srv1957015376=53, srv1774097822=45, srv1354360676=24, srv193600327=52, srv345037907=70, srv1149377527=6, srv130569168=22, srv597380146=82, srv220117556=67, srv2080486709=62, srv1162684455=8, srv124158037=16, srv1319696283=23, srv624286773=83, srv906086149=97, srv1763967513=44, srv1496212035=30, srv829830915=93, srv1613119843=39, srv1480275521=29, srv2039409182=60, srv1744225492=43, srv1297526899=21, srv549751938=80, srv2017293954=59, srv948514491=98, srv1154613002=7, srv1570627099=34, srv1976379829=54, srv2133666664=65, srv11950687=12, srv1256012365=17, srv1676510302=41, srv379015622=72, srv1976637450=55, srv112358043=5, srv1627214712=40, srv1370562333=25, srv849183247=96, srv1011335088=0, srv2102014795=63, srv776965031=90, srv1293742673=20, srv1216448736=13, srv1608303718=38, srv1072090062=4} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-24 02:53:19,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-24 02:53:19,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-24 02:53:19,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-24 02:53:19,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-24 02:53:19,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-24 02:53:19,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-24 02:53:19,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-24 02:53:19,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-24 02:53:19,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-24 02:53:19,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-24 02:53:19,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-24 02:53:19,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-24 02:53:19,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-24 02:53:19,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-24 02:53:19,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-24 02:53:19,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-24 02:53:19,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-24 02:53:19,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-24 02:53:19,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-24 02:53:19,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-24 02:53:19,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-24 02:53:19,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-24 02:53:19,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-24 02:53:19,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-24 02:53:19,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-24 02:53:19,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-24 02:53:19,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-24 02:53:19,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-24 02:53:19,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-24 02:53:19,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-24 02:53:19,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-24 02:53:19,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-24 02:53:19,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-24 02:53:19,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-24 02:53:19,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-24 02:53:19,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-24 02:53:19,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-24 02:53:19,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-24 02:53:19,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-24 02:53:19,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-24 02:53:19,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-24 02:53:19,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-24 02:53:19,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-24 02:53:19,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-24 02:53:19,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-24 02:53:19,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-24 02:53:19,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-24 02:53:19,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-24 02:53:19,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-24 02:53:19,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-24 02:53:19,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-24 02:53:19,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-24 02:53:19,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-24 02:53:19,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-24 02:53:19,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-24 02:53:19,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-24 02:53:19,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-24 02:53:19,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-24 02:53:19,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-24 02:53:19,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-24 02:53:19,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-24 02:53:19,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-24 02:53:19,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-24 02:53:19,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-24 02:53:19,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-24 02:53:19,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-24 02:53:19,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-24 02:53:19,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-24 02:53:19,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-24 02:53:19,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-24 02:53:19,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-24 02:53:19,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-24 02:53:19,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-24 02:53:19,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-24 02:53:19,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-24 02:53:19,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-24 02:53:19,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-24 02:53:19,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-24 02:53:19,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-24 02:53:19,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-24 02:53:19,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-24 02:53:19,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-24 02:53:19,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-24 02:53:19,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-24 02:53:19,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-24 02:53:19,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-24 02:53:19,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-24 02:53:19,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-24 02:53:19,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-24 02:53:19,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-24 02:53:19,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-24 02:53:19,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-24 02:53:19,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-24 02:53:19,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-24 02:53:19,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-24 02:53:19,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-24 02:53:19,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-24 02:53:19,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-24 02:53:19,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-24 02:53:19,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-24 02:53:19,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-24 02:53:19,559 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.9881631748512408E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0022721660362024225); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-24 02:53:19,559 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1676510302:83 , srv193600327:83 , srv1983557005:83 , srv2010153321:83 , srv2133666664:83 , srv509380634:83 , srv1162684455:84 , srv1165432180:84 , srv1216448736:84 , srv1256012365:84 , srv1260065762:84 , srv1297526899:84 , srv130569168:84 , srv1319696283:84 , srv1354360676:84 , srv1480275521:84 , srv1496212035:84 , srv1516060451:84 , srv1570627099:84 , srv1582266676:84 , srv1583063226:84 , srv1613119843:84 , srv1627214712:84 , srv1744225492:84 , srv1863877801:84 , srv1872599396:84 , srv1931432803:84 , srv1976379829:84 , srv1979763363:84 , srv2017293954:84 , srv2054911381:84 , srv2080486709:84 , srv2102014795:84 , srv220117556:84 , srv240754922:84 , srv345037907:84 , srv41152192:84 , srv458870246:84 , srv504257811:84 , srv519764500:84 , srv590134587:84 , srv597380146:84 , srv73485378:84 , srv804007425:84 , srv840247320:84 , srv1011335088:85 , srv1023258354:85 , srv1030237206:85 , srv1047537726:85 , srv1072090062:85 , srv112358043:85 , srv1149377527:85 , srv1154613002:85 , srv1176476804:85 , srv1192008256:85 , srv11950687:85 , srv1217420278:85 , srv1235115868:85 , srv1292387647:85 , srv1293742673:85 , srv1370562333:85 , srv1387580634:85 , srv1417434795:85 , srv1456738428:85 , srv1515118140:85 , srv1531307561:85 , srv1589494520:85 , srv1608303718:85 , srv1708255259:85 , srv1763967513:85 , srv1774097822:85 , srv1776098336:85 , srv1803674097:85 , srv1869856496:85 , srv1957015376:85 , srv1976637450:85 , srv2039409182:85 , srv2132233200:85 , srv2146419750:85 , srv237827580:85 , srv367697521:85 , srv379015622:85 , srv489930257:85 , srv503520151:85 , srv549751938:85 , srv624286773:85 , srv640932582:85 , srv704955788:85 , srv71818017:85 , srv76779757:85 , srv776965031:85 , srv820424968:85 , srv829830915:85 , srv848886138:85 , srv849183247:85 , srv906086149:85 , srv948514491:85 , srv994337915:85 , srv124158037:86 , srv669081196:635 }
2023-11-24 02:53:19,590 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=321 (was 321), ProcessCount=170 (was 170), AvailableMemoryMB=7299 (was 8108)
2023-11-24 02:53:19,600 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=321, ProcessCount=170, AvailableMemoryMB=7298
2023-11-24 02:53:19,601 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-24 02:53:19,601 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-11-24 02:53:19,613 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv10212560:84 , srv109380100:84 , srv1109158676:84 , srv1196058343:84 , srv1244179948:84 , srv130596158:84 , srv13111325:84 , srv13315794:84 , srv1492982419:84 , srv1533863098:84 , srv1589355642:84 , srv1608842153:84 , srv1663780380:84 , srv1698938826:84 , srv1843160445:84 , srv1862243286:84 , srv1873153395:84 , srv2061151747:84 , srv2075124476:84 , srv2107278787:84 , srv2135649310:84 , srv292956625:84 , srv314264365:84 , srv581048600:84 , srv627539356:84 , srv695559598:84 , srv744855358:84 , srv929989631:84 , srv981187568:84 , srv24996929:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-11-24 02:53:19,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-24 02:53:19,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1533863098=9, srv2061151747=17, srv2075124476=18, srv13111325=6, srv1608842153=11, srv581048600=24, srv627539356=25, srv1109158676=2, srv10212560=0, srv109380100=1, srv1698938826=13, srv1196058343=3, srv24996929=21, srv744855358=27, srv1873153395=16, srv1492982419=8, srv1862243286=15, srv314264365=23, srv695559598=26, srv1244179948=4, srv292956625=22, srv929989631=28, srv13315794=7, srv130596158=5, srv1589355642=10, srv1663780380=12, srv2107278787=19, srv1843160445=14, srv2135649310=20, srv981187568=29} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-24 02:53:19,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-24 02:53:19,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-24 02:53:19,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-24 02:53:19,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-24 02:53:19,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-24 02:53:19,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-24 02:53:19,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-24 02:53:19,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-24 02:53:19,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-24 02:53:19,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-24 02:53:19,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-24 02:53:19,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-24 02:53:19,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-24 02:53:19,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-24 02:53:19,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-24 02:53:19,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-24 02:53:19,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-24 02:53:19,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-24 02:53:19,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-24 02:53:19,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-24 02:53:19,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-24 02:53:19,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-24 02:53:19,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-24 02:53:19,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-24 02:53:19,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-24 02:53:19,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-24 02:53:19,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-24 02:53:19,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-24 02:53:19,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-24 02:53:19,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-24 02:53:19,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-11-24 02:53:19,653 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0744395119861376, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-24 02:53:19,654 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912427599735876, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0744395119861376, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-11-24 02:55:19,635 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6263386 different iterations.  Found a solution that moves 1846 regions; Going from a computed imbalance of 0.9912427599735876 to a new imbalance of 4.308656090753194E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6837037037037037, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-24 02:55:19,640 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv10212560:90 , srv109380100:90 , srv1109158676:90 , srv1196058343:90 , srv1244179948:90 , srv130596158:90 , srv13111325:90 , srv13315794:90 , srv1492982419:90 , srv1533863098:90 , srv1589355642:90 , srv1608842153:90 , srv1663780380:90 , srv1698938826:90 , srv1843160445:90 , srv1862243286:90 , srv1873153395:90 , srv2061151747:90 , srv2075124476:90 , srv2107278787:90 , srv2135649310:90 , srv24996929:90 , srv292956625:90 , srv314264365:90 , srv581048600:90 , srv627539356:90 , srv695559598:90 , srv744855358:90 , srv929989631:90 , srv981187568:90 }
2023-11-24 02:55:19,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-24 02:55:19,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1533863098=9, srv2061151747=17, srv2075124476=18, srv13111325=6, srv1608842153=11, srv581048600=24, srv627539356=25, srv1109158676=2, srv10212560=0, srv109380100=1, srv1698938826=13, srv1196058343=3, srv24996929=21, srv744855358=27, srv1873153395=16, srv1492982419=8, srv1862243286=15, srv314264365=23, srv695559598=26, srv1244179948=4, srv292956625=22, srv929989631=28, srv13315794=7, srv130596158=5, srv1589355642=10, srv1663780380=12, srv2107278787=19, srv1843160445=14, srv2135649310=20, srv981187568=29} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-24 02:55:19,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-24 02:55:19,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-24 02:55:19,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-24 02:55:19,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-24 02:55:19,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-24 02:55:19,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-24 02:55:19,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-24 02:55:19,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-24 02:55:19,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-24 02:55:19,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-24 02:55:19,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-24 02:55:19,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-24 02:55:19,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-24 02:55:19,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-24 02:55:19,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-24 02:55:19,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-24 02:55:19,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-24 02:55:19,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-24 02:55:19,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-24 02:55:19,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-24 02:55:19,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-24 02:55:19,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-24 02:55:19,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-24 02:55:19,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-24 02:55:19,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-24 02:55:19,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-24 02:55:19,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-24 02:55:19,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-24 02:55:19,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-24 02:55:19,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-24 02:55:19,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-11-24 02:55:19,658 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-24 02:55:19,658 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv10212560:90 , srv109380100:90 , srv1109158676:90 , srv1196058343:90 , srv1244179948:90 , srv130596158:90 , srv13111325:90 , srv13315794:90 , srv1492982419:90 , srv1533863098:90 , srv1589355642:90 , srv1608842153:90 , srv1663780380:90 , srv1698938826:90 , srv1843160445:90 , srv1862243286:90 , srv1873153395:90 , srv2061151747:90 , srv2075124476:90 , srv2107278787:90 , srv2135649310:90 , srv24996929:90 , srv292956625:90 , srv314264365:90 , srv581048600:90 , srv627539356:90 , srv695559598:90 , srv744855358:90 , srv929989631:90 , srv981187568:90 }
2023-11-24 02:55:19,670 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=190 (was 321), ProcessCount=170 (was 170), AvailableMemoryMB=6589 (was 7298)