2024-01-27 14:54:19,316 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2024-01-27 14:54:19,578 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2024-01-27 14:54:19,674 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2024-01-27 14:54:19,734 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-01-27 14:54:19,740 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-01-27 14:54:19,753 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=257, ProcessCount=176, AvailableMemoryMB=2275
2024-01-27 14:54:19,755 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-01-27 14:54:19,756 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-01-27 14:54:19,882 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1000690022:84 , srv1022192752:84 , srv1037866983:84 , srv1057332883:84 , srv1086550082:84 , srv1097593266:84 , srv1101806130:84 , srv111089164:84 , srv1119619480:84 , srv1164894978:84 , srv1205406239:84 , srv120571580:84 , srv1207428883:84 , srv1227614926:84 , srv1239370270:84 , srv1288267563:84 , srv1294175753:84 , srv1305401212:84 , srv1326052981:84 , srv1342838772:84 , srv1351281891:84 , srv1368322856:84 , srv1381241543:84 , srv1390779204:84 , srv1405659350:84 , srv1411901661:84 , srv1436378955:84 , srv1441462372:84 , srv1442047217:84 , srv1457297977:84 , srv1488656603:84 , srv1498701247:84 , srv1499370725:84 , srv1521954737:84 , srv1524235380:84 , srv1583487487:84 , srv1585360600:84 , srv1592920264:84 , srv1619043852:84 , srv1634214462:84 , srv1636521567:84 , srv1707668475:84 , srv1724517424:84 , srv1726067929:84 , srv1757227844:84 , srv17701272:84 , srv1779228860:84 , srv1802245420:84 , srv1808712252:84 , srv1817939588:84 , srv1842174449:84 , srv1874408797:84 , srv188960784:84 , srv190258366:84 , srv1939564573:84 , srv1942300805:84 , srv1959234502:84 , srv1999555419:84 , srv2010215225:84 , srv2021927291:84 , srv2047427911:84 , srv209585380:84 , srv2123122888:84 , srv2143407869:84 , srv216266841:84 , srv229544108:84 , srv256008705:84 , srv25896287:84 , srv300351387:84 , srv307097785:84 , srv3505747:84 , srv408101167:84 , srv418076260:84 , srv424438513:84 , srv437027669:84 , srv445967386:84 , srv482803835:84 , srv50768586:84 , srv516567068:84 , srv519646507:84 , srv538130100:84 , srv561831890:84 , srv568868418:84 , srv57972889:84 , srv585222574:84 , srv596097308:84 , srv618390122:84 , srv644622183:84 , srv750841658:84 , srv78891521:84 , srv812937961:84 , srv831374856:84 , srv851326621:84 , srv902885642:84 , srv905459751:84 , srv944623268:84 , srv957470074:84 , srv974060573:84 , srv997701489:84 , srv1210957635:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2024-01-27 14:54:19,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-27 14:54:19,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv831374856=92, srv229544108=66, srv812937961=91, srv1205406239=10, srv1636521567=41, srv1817939588=50, srv437027669=75, srv561831890=82, srv516567068=79, srv519646507=80, srv902885642=94, srv1326052981=19, srv1368322856=22, srv1411901661=26, srv585222574=85, srv974060573=98, srv2010215225=59, srv1488656603=31, srv1779228860=47, srv1619043852=39, srv25896287=68, srv1592920264=38, srv1874408797=52, srv1842174449=51, srv1524235380=35, srv1942300805=56, srv1288267563=16, srv1634214462=40, srv216266841=65, srv209585380=62, srv1294175753=17, srv596097308=86, srv1022192752=1, srv1305401212=18, srv188960784=53, srv1521954737=34, srv424438513=74, srv1405659350=25, srv445967386=76, srv1442047217=29, srv1097593266=5, srv2143407869=64, srv1457297977=30, srv2123122888=63, srv78891521=90, srv750841658=89, srv944623268=96, srv1757227844=45, srv1441462372=28, srv1499370725=33, srv1207428883=12, srv57972889=84, srv957470074=97, srv1037866983=2, srv1210957635=13, srv1583487487=36, srv17701272=46, srv1227614926=14, srv3505747=71, srv190258366=54, srv300351387=69, srv851326621=93, srv997701489=99, srv1498701247=32, srv1585360600=37, srv1164894978=9, srv568868418=83, srv1436378955=27, srv1999555419=58, srv1390779204=24, srv1707668475=42, srv644622183=88, srv1724517424=43, srv307097785=70, srv1808712252=49, srv905459751=95, srv111089164=7, srv538130100=81, srv1101806130=6, srv618390122=87, srv1939564573=55, srv1959234502=57, srv1000690022=0, srv1342838772=20, srv1119619480=8, srv120571580=11, srv1381241543=23, srv1057332883=3, srv50768586=78, srv1802245420=48, srv2047427911=61, srv1086550082=4, srv418076260=73, srv1239370270=15, srv256008705=67, srv408101167=72, srv2021927291=60, srv1351281891=21, srv1726067929=44, srv482803835=77} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-27 14:54:19,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-27 14:54:19,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-27 14:54:19,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-27 14:54:19,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-27 14:54:19,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-27 14:54:19,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-27 14:54:19,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-27 14:54:19,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-27 14:54:19,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-27 14:54:19,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-27 14:54:19,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-27 14:54:19,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-27 14:54:19,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-27 14:54:19,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-27 14:54:19,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-27 14:54:19,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-27 14:54:19,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-27 14:54:19,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-27 14:54:19,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-27 14:54:19,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-27 14:54:19,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-27 14:54:19,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-27 14:54:19,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-27 14:54:19,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-27 14:54:19,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-27 14:54:19,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-27 14:54:19,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-27 14:54:19,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-27 14:54:19,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-27 14:54:19,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-27 14:54:19,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-01-27 14:54:19,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-01-27 14:54:19,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-01-27 14:54:19,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-01-27 14:54:19,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-01-27 14:54:19,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-01-27 14:54:19,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-01-27 14:54:19,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-01-27 14:54:19,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-01-27 14:54:19,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-01-27 14:54:19,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-01-27 14:54:19,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-01-27 14:54:19,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-01-27 14:54:19,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-01-27 14:54:19,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-01-27 14:54:19,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-01-27 14:54:19,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-01-27 14:54:19,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-01-27 14:54:19,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-01-27 14:54:19,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-01-27 14:54:19,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-01-27 14:54:19,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-01-27 14:54:19,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-01-27 14:54:19,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-01-27 14:54:19,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-01-27 14:54:19,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-01-27 14:54:19,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-01-27 14:54:19,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-01-27 14:54:19,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-01-27 14:54:19,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-01-27 14:54:19,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-01-27 14:54:19,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-01-27 14:54:19,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-01-27 14:54:19,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-01-27 14:54:19,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-01-27 14:54:19,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-01-27 14:54:19,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-01-27 14:54:19,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-01-27 14:54:19,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-01-27 14:54:19,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-01-27 14:54:19,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-01-27 14:54:19,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-01-27 14:54:19,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-01-27 14:54:19,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-01-27 14:54:19,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-01-27 14:54:19,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-01-27 14:54:19,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-01-27 14:54:19,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-01-27 14:54:19,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-01-27 14:54:19,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-01-27 14:54:19,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-01-27 14:54:19,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-01-27 14:54:19,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-01-27 14:54:19,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-01-27 14:54:19,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-01-27 14:54:19,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-01-27 14:54:19,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-01-27 14:54:19,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-01-27 14:54:19,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-01-27 14:54:19,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-01-27 14:54:19,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-01-27 14:54:19,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-01-27 14:54:19,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-01-27 14:54:19,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-01-27 14:54:19,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-01-27 14:54:19,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-01-27 14:54:19,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-01-27 14:54:19,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-01-27 14:54:19,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-01-27 14:54:19,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-01-27 14:54:19,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-01-27 14:54:20,124 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-27 14:54:20,125 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-01-27 14:54:20,125 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-01-27 14:54:20,400 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 398 ms to try 5000 different iterations.  Found a solution that moves 4690 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1816079130724837. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.042624055134035936, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06035435948890002, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5211111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.042624055134035936, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17733333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23825, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-27 14:54:20,410 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1817939588:81 , srv1390779204:82 , srv424438513:82 , srv561831890:82 , srv644622183:82 , srv812937961:82 , srv1101806130:83 , srv1119619480:83 , srv1305401212:83 , srv1457297977:83 , srv1724517424:83 , srv1874408797:83 , srv1939564573:83 , srv300351387:83 , srv831374856:83 , srv1288267563:84 , srv1499370725:84 , srv1779228860:84 , srv3505747:84 , srv50768586:84 , srv596097308:84 , srv974060573:84 , srv1086550082:85 , srv120571580:85 , srv1368322856:85 , srv1441462372:85 , srv1521954737:85 , srv1583487487:85 , srv1592920264:85 , srv1634214462:85 , srv1757227844:85 , srv1959234502:85 , srv216266841:85 , srv418076260:85 , srv437027669:85 , srv445967386:85 , srv519646507:85 , srv538130100:85 , srv618390122:85 , srv750841658:85 , srv997701489:85 , srv1037866983:86 , srv1239370270:86 , srv1342838772:86 , srv1498701247:86 , srv1524235380:86 , srv1619043852:86 , srv1726067929:86 , srv188960784:86 , srv1942300805:86 , srv2123122888:86 , srv2143407869:86 , srv256008705:86 , srv408101167:86 , srv78891521:86 , srv851326621:86 , srv944623268:86 , srv957470074:86 , srv1000690022:87 , srv1057332883:87 , srv111089164:87 , srv1205406239:87 , srv1207428883:87 , srv1326052981:87 , srv1351281891:87 , srv1381241543:87 , srv1405659350:87 , srv1488656603:87 , srv1636521567:87 , srv1802245420:87 , srv2010215225:87 , srv209585380:87 , srv57972889:87 , srv585222574:87 , srv902885642:87 , srv1411901661:88 , srv1436378955:88 , srv17701272:88 , srv1842174449:88 , srv190258366:88 , srv1999555419:88 , srv25896287:88 , srv568868418:88 , srv1294175753:89 , srv1808712252:89 , srv229544108:89 , srv307097785:89 , srv905459751:89 , srv1164894978:90 , srv1227614926:90 , srv1442047217:90 , srv516567068:90 , srv1022192752:91 , srv1585360600:91 , srv2021927291:91 , srv2047427911:91 , srv482803835:91 , srv1707668475:92 , srv1097593266:93 , srv1210957635:469 }
2024-01-27 14:54:20,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-27 14:54:20,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv831374856=92, srv229544108=66, srv812937961=91, srv1205406239=10, srv1636521567=41, srv1817939588=50, srv437027669=75, srv561831890=82, srv516567068=79, srv519646507=80, srv902885642=94, srv1326052981=19, srv1368322856=22, srv1411901661=26, srv585222574=85, srv974060573=98, srv2010215225=59, srv1488656603=31, srv1779228860=47, srv1619043852=39, srv25896287=68, srv1592920264=38, srv1874408797=52, srv1842174449=51, srv1524235380=35, srv1942300805=56, srv1288267563=16, srv1634214462=40, srv216266841=65, srv209585380=62, srv1294175753=17, srv596097308=86, srv1022192752=1, srv1305401212=18, srv188960784=53, srv1521954737=34, srv424438513=74, srv1405659350=25, srv445967386=76, srv1442047217=29, srv1097593266=5, srv2143407869=64, srv1457297977=30, srv2123122888=63, srv78891521=90, srv750841658=89, srv944623268=96, srv1757227844=45, srv1441462372=28, srv1499370725=33, srv1207428883=12, srv57972889=84, srv957470074=97, srv1037866983=2, srv1210957635=13, srv1583487487=36, srv17701272=46, srv1227614926=14, srv3505747=71, srv190258366=54, srv300351387=69, srv851326621=93, srv997701489=99, srv1498701247=32, srv1585360600=37, srv1164894978=9, srv568868418=83, srv1436378955=27, srv1999555419=58, srv1390779204=24, srv1707668475=42, srv644622183=88, srv1724517424=43, srv307097785=70, srv1808712252=49, srv905459751=95, srv111089164=7, srv538130100=81, srv1101806130=6, srv618390122=87, srv1939564573=55, srv1959234502=57, srv1000690022=0, srv1342838772=20, srv1119619480=8, srv120571580=11, srv1381241543=23, srv1057332883=3, srv50768586=78, srv1802245420=48, srv2047427911=61, srv1086550082=4, srv418076260=73, srv1239370270=15, srv256008705=67, srv408101167=72, srv2021927291=60, srv1351281891=21, srv1726067929=44, srv482803835=77} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-27 14:54:20,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-27 14:54:20,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-27 14:54:20,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-27 14:54:20,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-27 14:54:20,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-27 14:54:20,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-27 14:54:20,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-27 14:54:20,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-27 14:54:20,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-27 14:54:20,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-27 14:54:20,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-27 14:54:20,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-27 14:54:20,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-27 14:54:20,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-27 14:54:20,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-27 14:54:20,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-27 14:54:20,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-27 14:54:20,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-27 14:54:20,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-27 14:54:20,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-27 14:54:20,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-27 14:54:20,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-27 14:54:20,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-27 14:54:20,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-27 14:54:20,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-27 14:54:20,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-27 14:54:20,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-27 14:54:20,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-27 14:54:20,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-27 14:54:20,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-27 14:54:20,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-01-27 14:54:20,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-01-27 14:54:20,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-01-27 14:54:20,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-01-27 14:54:20,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-01-27 14:54:20,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-01-27 14:54:20,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-01-27 14:54:20,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-01-27 14:54:20,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-01-27 14:54:20,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-01-27 14:54:20,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-01-27 14:54:20,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-01-27 14:54:20,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-01-27 14:54:20,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-01-27 14:54:20,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-01-27 14:54:20,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-01-27 14:54:20,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-01-27 14:54:20,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-01-27 14:54:20,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-01-27 14:54:20,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-01-27 14:54:20,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-01-27 14:54:20,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-01-27 14:54:20,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-01-27 14:54:20,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-01-27 14:54:20,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-01-27 14:54:20,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-01-27 14:54:20,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-01-27 14:54:20,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-01-27 14:54:20,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-01-27 14:54:20,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-01-27 14:54:20,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-01-27 14:54:20,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-01-27 14:54:20,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-01-27 14:54:20,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-01-27 14:54:20,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-01-27 14:54:20,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-01-27 14:54:20,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-01-27 14:54:20,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-01-27 14:54:20,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-01-27 14:54:20,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-01-27 14:54:20,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-01-27 14:54:20,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-01-27 14:54:20,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-01-27 14:54:20,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-01-27 14:54:20,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-01-27 14:54:20,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-01-27 14:54:20,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-01-27 14:54:20,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-01-27 14:54:20,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-01-27 14:54:20,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-01-27 14:54:20,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-01-27 14:54:20,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-01-27 14:54:20,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-01-27 14:54:20,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-01-27 14:54:20,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-01-27 14:54:20,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-01-27 14:54:20,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-01-27 14:54:20,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-01-27 14:54:20,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-01-27 14:54:20,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-01-27 14:54:20,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-01-27 14:54:20,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-01-27 14:54:20,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-01-27 14:54:20,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-01-27 14:54:20,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-01-27 14:54:20,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-01-27 14:54:20,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-01-27 14:54:20,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-01-27 14:54:20,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-01-27 14:54:20,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-01-27 14:54:20,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-01-27 14:54:20,596 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.042624055134035936, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06035435948890002, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.042624055134035936, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17733333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23825, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-27 14:54:20,596 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-01-27 14:54:20,597 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18157507299057857, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.042624055134035936, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06035435948890002, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.042624055134035936, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17733333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23825, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-01-27 14:54:20,851 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 381 ms to try 5000 different iterations.  Found a solution that moves 2613 regions; Going from a computed imbalance of 0.18157507299057857 to a new imbalance of 0.013278463779635694. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02261989849570736); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03326255111078082, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29033333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02261989849570736); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.009916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04525, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-27 14:54:20,862 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1097593266:78 , srv1707668475:78 , srv2047427911:79 , srv1585360600:80 , srv2021927291:80 , srv1022192752:81 , srv1442047217:81 , srv229544108:81 , srv482803835:81 , srv905459751:81 , srv1057332883:82 , srv1164894978:82 , srv1227614926:82 , srv1411901661:82 , srv17701272:82 , srv1802245420:82 , srv307097785:82 , srv516567068:82 , srv1294175753:83 , srv1405659350:83 , srv1037866983:84 , srv1205406239:84 , srv1436378955:84 , srv1636521567:84 , srv1808712252:84 , srv190258366:84 , srv25896287:84 , srv408101167:84 , srv568868418:84 , srv902885642:84 , srv957470074:84 , srv1000690022:85 , srv1207428883:85 , srv1239370270:85 , srv1368322856:85 , srv1488656603:85 , srv1498701247:85 , srv1583487487:85 , srv1842174449:85 , srv216266841:85 , srv57972889:85 , srv78891521:85 , srv1326052981:86 , srv1342838772:86 , srv1381241543:86 , srv1634214462:86 , srv1726067929:86 , srv1999555419:86 , srv2010215225:86 , srv209585380:86 , srv2123122888:86 , srv256008705:86 , srv3505747:86 , srv445967386:86 , srv585222574:86 , srv596097308:86 , srv618390122:86 , srv944623268:86 , srv997701489:86 , srv111089164:87 , srv1288267563:87 , srv1441462372:87 , srv1524235380:87 , srv1619043852:87 , srv188960784:87 , srv1942300805:87 , srv2143407869:87 , srv300351387:87 , srv519646507:87 , srv851326621:87 , srv1086550082:88 , srv1499370725:88 , srv1521954737:88 , srv1592920264:88 , srv1757227844:88 , srv1959234502:88 , srv50768586:88 , srv750841658:88 , srv974060573:88 , srv1305401212:89 , srv1351281891:89 , srv1817939588:89 , srv1939564573:89 , srv418076260:89 , srv424438513:89 , srv437027669:89 , srv1101806130:90 , srv1119619480:90 , srv1779228860:90 , srv1874408797:90 , srv644622183:90 , srv812937961:90 , srv120571580:91 , srv1390779204:91 , srv1724517424:91 , srv561831890:91 , srv831374856:91 , srv1457297977:92 , srv538130100:92 , srv1210957635:506 }
2024-01-27 14:54:20,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-27 14:54:20,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv831374856=92, srv229544108=66, srv812937961=91, srv1205406239=10, srv1636521567=41, srv1817939588=50, srv437027669=75, srv561831890=82, srv516567068=79, srv519646507=80, srv902885642=94, srv1326052981=19, srv1368322856=22, srv1411901661=26, srv585222574=85, srv974060573=98, srv2010215225=59, srv1488656603=31, srv1779228860=47, srv1619043852=39, srv25896287=68, srv1592920264=38, srv1874408797=52, srv1842174449=51, srv1524235380=35, srv1942300805=56, srv1288267563=16, srv1634214462=40, srv216266841=65, srv209585380=62, srv1294175753=17, srv596097308=86, srv1022192752=1, srv1305401212=18, srv188960784=53, srv1521954737=34, srv424438513=74, srv1405659350=25, srv445967386=76, srv1442047217=29, srv1097593266=5, srv2143407869=64, srv1457297977=30, srv2123122888=63, srv78891521=90, srv750841658=89, srv944623268=96, srv1757227844=45, srv1441462372=28, srv1499370725=33, srv1207428883=12, srv57972889=84, srv957470074=97, srv1037866983=2, srv1210957635=13, srv1583487487=36, srv17701272=46, srv1227614926=14, srv3505747=71, srv190258366=54, srv300351387=69, srv851326621=93, srv997701489=99, srv1498701247=32, srv1585360600=37, srv1164894978=9, srv568868418=83, srv1436378955=27, srv1999555419=58, srv1390779204=24, srv1707668475=42, srv644622183=88, srv1724517424=43, srv307097785=70, srv1808712252=49, srv905459751=95, srv111089164=7, srv538130100=81, srv1101806130=6, srv618390122=87, srv1939564573=55, srv1959234502=57, srv1000690022=0, srv1342838772=20, srv1119619480=8, srv120571580=11, srv1381241543=23, srv1057332883=3, srv50768586=78, srv1802245420=48, srv2047427911=61, srv1086550082=4, srv418076260=73, srv1239370270=15, srv256008705=67, srv408101167=72, srv2021927291=60, srv1351281891=21, srv1726067929=44, srv482803835=77} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-27 14:54:20,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-27 14:54:20,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-27 14:54:20,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-27 14:54:20,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-27 14:54:20,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-27 14:54:20,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-27 14:54:20,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-27 14:54:20,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-27 14:54:20,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-27 14:54:20,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-27 14:54:20,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-27 14:54:20,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-27 14:54:20,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-27 14:54:20,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-27 14:54:20,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-27 14:54:20,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-27 14:54:20,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-27 14:54:20,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-27 14:54:20,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-27 14:54:20,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-27 14:54:20,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-27 14:54:20,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-27 14:54:20,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-27 14:54:20,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-27 14:54:20,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-27 14:54:20,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-27 14:54:20,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-27 14:54:20,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-27 14:54:20,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-27 14:54:20,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-27 14:54:20,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-01-27 14:54:20,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-01-27 14:54:20,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-01-27 14:54:20,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-01-27 14:54:20,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-01-27 14:54:20,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-01-27 14:54:20,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-01-27 14:54:20,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-01-27 14:54:20,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-01-27 14:54:20,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-01-27 14:54:20,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-01-27 14:54:20,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-01-27 14:54:20,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-01-27 14:54:20,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-01-27 14:54:20,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-01-27 14:54:20,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-01-27 14:54:20,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-01-27 14:54:20,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-01-27 14:54:20,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-01-27 14:54:20,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-01-27 14:54:20,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-01-27 14:54:20,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-01-27 14:54:20,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-01-27 14:54:20,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-01-27 14:54:20,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-01-27 14:54:20,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-01-27 14:54:20,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-01-27 14:54:20,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-01-27 14:54:20,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-01-27 14:54:20,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-01-27 14:54:20,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-01-27 14:54:20,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-01-27 14:54:20,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-01-27 14:54:20,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-01-27 14:54:20,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-01-27 14:54:20,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-01-27 14:54:20,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-01-27 14:54:20,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-01-27 14:54:20,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-01-27 14:54:20,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-01-27 14:54:20,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-01-27 14:54:20,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-01-27 14:54:20,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-01-27 14:54:20,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-01-27 14:54:20,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-01-27 14:54:20,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-01-27 14:54:20,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-01-27 14:54:20,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-01-27 14:54:20,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-01-27 14:54:20,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-01-27 14:54:20,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-01-27 14:54:20,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-01-27 14:54:20,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-01-27 14:54:20,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-01-27 14:54:20,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-01-27 14:54:20,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-01-27 14:54:20,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-01-27 14:54:20,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-01-27 14:54:20,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-01-27 14:54:20,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-01-27 14:54:20,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-01-27 14:54:20,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-01-27 14:54:20,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-01-27 14:54:20,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-01-27 14:54:20,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-01-27 14:54:20,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-01-27 14:54:20,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-01-27 14:54:20,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-01-27 14:54:20,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-01-27 14:54:20,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-01-27 14:54:20,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-01-27 14:54:21,018 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02261989849570736); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03326255111078082, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02261989849570736); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.009916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04525, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-27 14:54:21,018 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-01-27 14:54:21,019 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.013260167162574258, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02261989849570736); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03326255111078082, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02261989849570736); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.009916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04525, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-01-27 14:54:21,252 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 338 ms to try 5000 different iterations.  Found a solution that moves 981 regions; Going from a computed imbalance of 0.013260167162574258 to a new imbalance of 9.685577432447341E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0077977937092175405); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008632670957261212); MoveCostFunction : (multiplier=7.0, imbalance=0.109, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0077977937092175405); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.0E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004833333333333334); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-27 14:54:21,255 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv538130100:79 , srv120571580:81 , srv1351281891:82 , srv1457297977:82 , srv418076260:82 , srv111089164:83 , srv1119619480:83 , srv1592920264:83 , srv1619043852:83 , srv1724517424:83 , srv1757227844:83 , srv1808712252:83 , srv1999555419:83 , srv437027669:83 , srv1101806130:84 , srv1164894978:84 , srv1381241543:84 , srv1390779204:84 , srv1521954737:84 , srv1779228860:84 , srv1874408797:84 , srv188960784:84 , srv1959234502:84 , srv2143407869:84 , srv561831890:84 , srv585222574:84 , srv851326621:84 , srv1000690022:85 , srv1086550082:85 , srv1205406239:85 , srv1294175753:85 , srv1342838772:85 , srv1436378955:85 , srv1499370725:85 , srv1524235380:85 , srv1726067929:85 , srv190258366:85 , srv1939564573:85 , srv2010215225:85 , srv209585380:85 , srv256008705:85 , srv25896287:85 , srv482803835:85 , srv50768586:85 , srv568868418:85 , srv57972889:85 , srv618390122:85 , srv750841658:85 , srv812937961:85 , srv831374856:85 , srv944623268:85 , srv974060573:85 , srv1022192752:86 , srv1207428883:86 , srv1305401212:86 , srv1326052981:86 , srv1441462372:86 , srv1442047217:86 , srv1488656603:86 , srv1585360600:86 , srv1636521567:86 , srv1842174449:86 , srv1942300805:86 , srv2021927291:86 , srv2047427911:86 , srv2123122888:86 , srv307097785:86 , srv424438513:86 , srv516567068:86 , srv596097308:86 , srv644622183:86 , srv78891521:86 , srv902885642:86 , srv997701489:86 , srv1037866983:87 , srv1097593266:87 , srv1227614926:87 , srv1239370270:87 , srv1288267563:87 , srv1368322856:87 , srv1405659350:87 , srv1411901661:87 , srv1634214462:87 , srv17701272:87 , srv1817939588:87 , srv216266841:87 , srv229544108:87 , srv300351387:87 , srv3505747:87 , srv408101167:87 , srv519646507:87 , srv905459751:87 , srv1498701247:88 , srv1583487487:88 , srv1707668475:88 , srv445967386:88 , srv957470074:88 , srv1057332883:89 , srv1802245420:91 , srv1210957635:552 }
2024-01-27 14:54:21,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-27 14:54:21,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv831374856=92, srv229544108=66, srv812937961=91, srv1205406239=10, srv1636521567=41, srv1817939588=50, srv437027669=75, srv561831890=82, srv516567068=79, srv519646507=80, srv902885642=94, srv1326052981=19, srv1368322856=22, srv1411901661=26, srv585222574=85, srv974060573=98, srv2010215225=59, srv1488656603=31, srv1779228860=47, srv1619043852=39, srv25896287=68, srv1592920264=38, srv1874408797=52, srv1842174449=51, srv1524235380=35, srv1942300805=56, srv1288267563=16, srv1634214462=40, srv216266841=65, srv209585380=62, srv1294175753=17, srv596097308=86, srv1022192752=1, srv1305401212=18, srv188960784=53, srv1521954737=34, srv424438513=74, srv1405659350=25, srv445967386=76, srv1442047217=29, srv1097593266=5, srv2143407869=64, srv1457297977=30, srv2123122888=63, srv78891521=90, srv750841658=89, srv944623268=96, srv1757227844=45, srv1441462372=28, srv1499370725=33, srv1207428883=12, srv57972889=84, srv957470074=97, srv1037866983=2, srv1210957635=13, srv1583487487=36, srv17701272=46, srv1227614926=14, srv3505747=71, srv190258366=54, srv300351387=69, srv851326621=93, srv997701489=99, srv1498701247=32, srv1585360600=37, srv1164894978=9, srv568868418=83, srv1436378955=27, srv1999555419=58, srv1390779204=24, srv1707668475=42, srv644622183=88, srv1724517424=43, srv307097785=70, srv1808712252=49, srv905459751=95, srv111089164=7, srv538130100=81, srv1101806130=6, srv618390122=87, srv1939564573=55, srv1959234502=57, srv1000690022=0, srv1342838772=20, srv1119619480=8, srv120571580=11, srv1381241543=23, srv1057332883=3, srv50768586=78, srv1802245420=48, srv2047427911=61, srv1086550082=4, srv418076260=73, srv1239370270=15, srv256008705=67, srv408101167=72, srv2021927291=60, srv1351281891=21, srv1726067929=44, srv482803835=77} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-27 14:54:21,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-27 14:54:21,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-27 14:54:21,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-27 14:54:21,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-27 14:54:21,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-27 14:54:21,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-27 14:54:21,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-27 14:54:21,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-27 14:54:21,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-27 14:54:21,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-27 14:54:21,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-27 14:54:21,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-27 14:54:21,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-27 14:54:21,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-27 14:54:21,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-27 14:54:21,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-27 14:54:21,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-27 14:54:21,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-27 14:54:21,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-27 14:54:21,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-27 14:54:21,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-27 14:54:21,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-27 14:54:21,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-27 14:54:21,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-27 14:54:21,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-27 14:54:21,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-27 14:54:21,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-27 14:54:21,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-27 14:54:21,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-27 14:54:21,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-27 14:54:21,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-01-27 14:54:21,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-01-27 14:54:21,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-01-27 14:54:21,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-01-27 14:54:21,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-01-27 14:54:21,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-01-27 14:54:21,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-01-27 14:54:21,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-01-27 14:54:21,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-01-27 14:54:21,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-01-27 14:54:21,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-01-27 14:54:21,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-01-27 14:54:21,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-01-27 14:54:21,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-01-27 14:54:21,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-01-27 14:54:21,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-01-27 14:54:21,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-01-27 14:54:21,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-01-27 14:54:21,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-01-27 14:54:21,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-01-27 14:54:21,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-01-27 14:54:21,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-01-27 14:54:21,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-01-27 14:54:21,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-01-27 14:54:21,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-01-27 14:54:21,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-01-27 14:54:21,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-01-27 14:54:21,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-01-27 14:54:21,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-01-27 14:54:21,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-01-27 14:54:21,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-01-27 14:54:21,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-01-27 14:54:21,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-01-27 14:54:21,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-01-27 14:54:21,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-01-27 14:54:21,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-01-27 14:54:21,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-01-27 14:54:21,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-01-27 14:54:21,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-01-27 14:54:21,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-01-27 14:54:21,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-01-27 14:54:21,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-01-27 14:54:21,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-01-27 14:54:21,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-01-27 14:54:21,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-01-27 14:54:21,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-01-27 14:54:21,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-01-27 14:54:21,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-01-27 14:54:21,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-01-27 14:54:21,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-01-27 14:54:21,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-01-27 14:54:21,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-01-27 14:54:21,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-01-27 14:54:21,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-01-27 14:54:21,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-01-27 14:54:21,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-01-27 14:54:21,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-01-27 14:54:21,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-01-27 14:54:21,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-01-27 14:54:21,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-01-27 14:54:21,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-01-27 14:54:21,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-01-27 14:54:21,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-01-27 14:54:21,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-01-27 14:54:21,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-01-27 14:54:21,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-01-27 14:54:21,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-01-27 14:54:21,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-01-27 14:54:21,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-01-27 14:54:21,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-01-27 14:54:21,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-01-27 14:54:21,409 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0077977937092175405); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008632670957261212); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0077977937092175405); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.0E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004833333333333334); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-27 14:54:21,409 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-01-27 14:54:21,410 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=9.616886344283275E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0077977937092175405); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008632670957261212); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0077977937092175405); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.0E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004833333333333334); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-01-27 14:54:21,731 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 428 ms to try 5000 different iterations.  Found a solution that moves 281 regions; Going from a computed imbalance of 9.616886344283275E-4 to a new imbalance of 1.3021040299828485E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.23782800878924E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.03122222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.23782800878924E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-27 14:54:21,732 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1802245420:82 , srv1227614926:83 , srv1326052981:83 , srv1498701247:83 , srv1842174449:83 , srv1942300805:83 , srv519646507:83 , srv831374856:83 , srv1022192752:84 , srv1057332883:84 , srv1086550082:84 , srv1097593266:84 , srv1207428883:84 , srv1239370270:84 , srv1288267563:84 , srv1342838772:84 , srv1351281891:84 , srv1441462372:84 , srv1488656603:84 , srv1499370725:84 , srv1524235380:84 , srv1583487487:84 , srv1634214462:84 , srv1779228860:84 , srv1817939588:84 , srv1959234502:84 , srv209585380:84 , srv2123122888:84 , srv216266841:84 , srv445967386:84 , srv482803835:84 , srv516567068:84 , srv585222574:84 , srv750841658:84 , srv905459751:84 , srv957470074:84 , srv1000690022:85 , srv1037866983:85 , srv1101806130:85 , srv111089164:85 , srv120571580:85 , srv1294175753:85 , srv1305401212:85 , srv1368322856:85 , srv1381241543:85 , srv1390779204:85 , srv1405659350:85 , srv1411901661:85 , srv1436378955:85 , srv1442047217:85 , srv1457297977:85 , srv1585360600:85 , srv1636521567:85 , srv1707668475:85 , srv1724517424:85 , srv1726067929:85 , srv17701272:85 , srv1808712252:85 , srv1874408797:85 , srv188960784:85 , srv190258366:85 , srv1939564573:85 , srv1999555419:85 , srv2010215225:85 , srv2021927291:85 , srv2047427911:85 , srv2143407869:85 , srv229544108:85 , srv256008705:85 , srv25896287:85 , srv300351387:85 , srv3505747:85 , srv418076260:85 , srv424438513:85 , srv437027669:85 , srv50768586:85 , srv561831890:85 , srv568868418:85 , srv57972889:85 , srv644622183:85 , srv851326621:85 , srv902885642:85 , srv944623268:85 , srv974060573:85 , srv997701489:85 , srv1119619480:86 , srv1164894978:86 , srv1205406239:86 , srv1521954737:86 , srv1619043852:86 , srv1757227844:86 , srv307097785:86 , srv408101167:86 , srv538130100:86 , srv596097308:86 , srv618390122:86 , srv78891521:86 , srv812937961:86 , srv1592920264:87 , srv1210957635:615 }
2024-01-27 14:54:21,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-27 14:54:21,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv831374856=92, srv229544108=66, srv812937961=91, srv1205406239=10, srv1636521567=41, srv1817939588=50, srv437027669=75, srv561831890=82, srv516567068=79, srv519646507=80, srv902885642=94, srv1326052981=19, srv1368322856=22, srv1411901661=26, srv585222574=85, srv974060573=98, srv2010215225=59, srv1488656603=31, srv1779228860=47, srv1619043852=39, srv25896287=68, srv1592920264=38, srv1874408797=52, srv1842174449=51, srv1524235380=35, srv1942300805=56, srv1288267563=16, srv1634214462=40, srv216266841=65, srv209585380=62, srv1294175753=17, srv596097308=86, srv1022192752=1, srv1305401212=18, srv188960784=53, srv1521954737=34, srv424438513=74, srv1405659350=25, srv445967386=76, srv1442047217=29, srv1097593266=5, srv2143407869=64, srv1457297977=30, srv2123122888=63, srv78891521=90, srv750841658=89, srv944623268=96, srv1757227844=45, srv1441462372=28, srv1499370725=33, srv1207428883=12, srv57972889=84, srv957470074=97, srv1037866983=2, srv1210957635=13, srv1583487487=36, srv17701272=46, srv1227614926=14, srv3505747=71, srv190258366=54, srv300351387=69, srv851326621=93, srv997701489=99, srv1498701247=32, srv1585360600=37, srv1164894978=9, srv568868418=83, srv1436378955=27, srv1999555419=58, srv1390779204=24, srv1707668475=42, srv644622183=88, srv1724517424=43, srv307097785=70, srv1808712252=49, srv905459751=95, srv111089164=7, srv538130100=81, srv1101806130=6, srv618390122=87, srv1939564573=55, srv1959234502=57, srv1000690022=0, srv1342838772=20, srv1119619480=8, srv120571580=11, srv1381241543=23, srv1057332883=3, srv50768586=78, srv1802245420=48, srv2047427911=61, srv1086550082=4, srv418076260=73, srv1239370270=15, srv256008705=67, srv408101167=72, srv2021927291=60, srv1351281891=21, srv1726067929=44, srv482803835=77} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-27 14:54:21,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-27 14:54:21,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-27 14:54:21,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-27 14:54:21,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-27 14:54:21,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-27 14:54:21,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-27 14:54:21,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-27 14:54:21,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-27 14:54:21,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-27 14:54:21,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-27 14:54:21,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-27 14:54:21,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-27 14:54:21,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-27 14:54:21,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-27 14:54:21,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-27 14:54:21,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-27 14:54:21,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-27 14:54:21,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-27 14:54:21,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-27 14:54:21,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-27 14:54:21,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-27 14:54:21,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-27 14:54:21,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-27 14:54:21,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-27 14:54:21,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-27 14:54:21,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-27 14:54:21,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-27 14:54:21,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-27 14:54:21,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-27 14:54:21,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-27 14:54:21,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-01-27 14:54:21,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-01-27 14:54:21,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-01-27 14:54:21,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-01-27 14:54:21,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-01-27 14:54:21,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-01-27 14:54:21,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-01-27 14:54:21,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-01-27 14:54:21,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-01-27 14:54:21,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-01-27 14:54:21,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-01-27 14:54:21,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-01-27 14:54:21,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-01-27 14:54:21,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-01-27 14:54:21,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-01-27 14:54:21,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-01-27 14:54:21,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-01-27 14:54:21,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-01-27 14:54:21,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-01-27 14:54:21,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-01-27 14:54:21,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-01-27 14:54:21,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-01-27 14:54:21,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-01-27 14:54:21,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-01-27 14:54:21,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-01-27 14:54:21,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-01-27 14:54:21,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-01-27 14:54:21,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-01-27 14:54:21,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-01-27 14:54:21,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-01-27 14:54:21,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-01-27 14:54:21,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-01-27 14:54:21,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-01-27 14:54:21,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-01-27 14:54:21,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-01-27 14:54:21,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-01-27 14:54:21,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-01-27 14:54:21,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-01-27 14:54:21,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-01-27 14:54:21,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-01-27 14:54:21,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-01-27 14:54:21,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-01-27 14:54:21,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-01-27 14:54:21,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-01-27 14:54:21,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-01-27 14:54:21,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-01-27 14:54:21,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-01-27 14:54:21,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-01-27 14:54:21,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-01-27 14:54:21,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-01-27 14:54:21,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-01-27 14:54:21,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-01-27 14:54:21,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-01-27 14:54:21,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-01-27 14:54:21,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-01-27 14:54:21,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-01-27 14:54:21,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-01-27 14:54:21,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-01-27 14:54:21,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-01-27 14:54:21,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-01-27 14:54:21,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-01-27 14:54:21,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-01-27 14:54:21,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-01-27 14:54:21,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-01-27 14:54:21,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-01-27 14:54:21,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-01-27 14:54:21,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-01-27 14:54:21,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-01-27 14:54:21,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-01-27 14:54:21,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-01-27 14:54:21,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-01-27 14:54:21,881 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.105343624538377E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.23782800878924E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.23782800878924E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-27 14:54:21,882 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1802245420:82 , srv1227614926:83 , srv1326052981:83 , srv1498701247:83 , srv1842174449:83 , srv1942300805:83 , srv519646507:83 , srv831374856:83 , srv1022192752:84 , srv1057332883:84 , srv1086550082:84 , srv1097593266:84 , srv1207428883:84 , srv1239370270:84 , srv1288267563:84 , srv1342838772:84 , srv1351281891:84 , srv1441462372:84 , srv1488656603:84 , srv1499370725:84 , srv1524235380:84 , srv1583487487:84 , srv1634214462:84 , srv1779228860:84 , srv1817939588:84 , srv1959234502:84 , srv209585380:84 , srv2123122888:84 , srv216266841:84 , srv445967386:84 , srv482803835:84 , srv516567068:84 , srv585222574:84 , srv750841658:84 , srv905459751:84 , srv957470074:84 , srv1000690022:85 , srv1037866983:85 , srv1101806130:85 , srv111089164:85 , srv120571580:85 , srv1294175753:85 , srv1305401212:85 , srv1368322856:85 , srv1381241543:85 , srv1390779204:85 , srv1405659350:85 , srv1411901661:85 , srv1436378955:85 , srv1442047217:85 , srv1457297977:85 , srv1585360600:85 , srv1636521567:85 , srv1707668475:85 , srv1724517424:85 , srv1726067929:85 , srv17701272:85 , srv1808712252:85 , srv1874408797:85 , srv188960784:85 , srv190258366:85 , srv1939564573:85 , srv1999555419:85 , srv2010215225:85 , srv2021927291:85 , srv2047427911:85 , srv2143407869:85 , srv229544108:85 , srv256008705:85 , srv25896287:85 , srv300351387:85 , srv3505747:85 , srv418076260:85 , srv424438513:85 , srv437027669:85 , srv50768586:85 , srv561831890:85 , srv568868418:85 , srv57972889:85 , srv644622183:85 , srv851326621:85 , srv902885642:85 , srv944623268:85 , srv974060573:85 , srv997701489:85 , srv1119619480:86 , srv1164894978:86 , srv1205406239:86 , srv1521954737:86 , srv1619043852:86 , srv1757227844:86 , srv307097785:86 , srv408101167:86 , srv538130100:86 , srv596097308:86 , srv618390122:86 , srv78891521:86 , srv812937961:86 , srv1592920264:87 , srv1210957635:615 }
2024-01-27 14:54:21,910 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=257 (was 257), ProcessCount=176 (was 176), AvailableMemoryMB=1684 (was 2275)
2024-01-27 14:54:21,918 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=257, ProcessCount=176, AvailableMemoryMB=1683
2024-01-27 14:54:21,919 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-01-27 14:54:21,919 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2024-01-27 14:54:21,930 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1002329352:84 , srv1015722041:84 , srv1125976871:84 , srv112804577:84 , srv1171160567:84 , srv1192374035:84 , srv1193261949:84 , srv1195616860:84 , srv1430096690:84 , srv1438141222:84 , srv1473292856:84 , srv1475594564:84 , srv1532251581:84 , srv1592347826:84 , srv1718027539:84 , srv1749195524:84 , srv1793361803:84 , srv1821492877:84 , srv1857802255:84 , srv1896180909:84 , srv2131922745:84 , srv225109012:84 , srv307889131:84 , srv3203305:84 , srv435193899:84 , srv560236243:84 , srv612141355:84 , srv7106960:84 , srv975343400:84 , srv115675271:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2024-01-27 14:54:21,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-27 14:54:21,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv975343400=29, srv1473292856=11, srv1125976871=2, srv1592347826=14, srv1193261949=7, srv1857802255=19, srv1896180909=20, srv3203305=24, srv1749195524=16, srv1430096690=9, srv1718027539=15, srv1475594564=12, srv307889131=23, srv115675271=4, srv1532251581=13, srv560236243=26, srv112804577=3, srv1195616860=8, srv1438141222=10, srv1821492877=18, srv435193899=25, srv1015722041=1, srv225109012=22, srv2131922745=21, srv7106960=28, srv1171160567=5, srv612141355=27, srv1192374035=6, srv1793361803=17, srv1002329352=0} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-27 14:54:21,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-27 14:54:21,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-27 14:54:21,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-27 14:54:21,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-27 14:54:21,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-27 14:54:21,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-27 14:54:21,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-27 14:54:21,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-27 14:54:21,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-27 14:54:21,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-27 14:54:21,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-27 14:54:21,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-27 14:54:21,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-27 14:54:21,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-27 14:54:21,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-27 14:54:21,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-27 14:54:21,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-27 14:54:21,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-27 14:54:21,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-27 14:54:21,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-27 14:54:21,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-27 14:54:21,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-27 14:54:21,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-27 14:54:21,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-27 14:54:21,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-27 14:54:21,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-27 14:54:21,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-27 14:54:21,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-27 14:54:21,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-27 14:54:21,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-27 14:54:21,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-01-27 14:54:21,972 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1286275816750457, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-27 14:54:21,973 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912598344574061, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1286275816750457, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2024-01-27 14:56:21,952 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 9693097 different iterations.  Found a solution that moves 1841 regions; Going from a computed imbalance of 0.9912598344574061 to a new imbalance of 4.296985841319951E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6818518518518518, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-27 14:56:21,957 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1002329352:90 , srv1015722041:90 , srv1125976871:90 , srv112804577:90 , srv115675271:90 , srv1171160567:90 , srv1192374035:90 , srv1193261949:90 , srv1195616860:90 , srv1430096690:90 , srv1438141222:90 , srv1473292856:90 , srv1475594564:90 , srv1532251581:90 , srv1592347826:90 , srv1718027539:90 , srv1749195524:90 , srv1793361803:90 , srv1821492877:90 , srv1857802255:90 , srv1896180909:90 , srv2131922745:90 , srv225109012:90 , srv307889131:90 , srv3203305:90 , srv435193899:90 , srv560236243:90 , srv612141355:90 , srv7106960:90 , srv975343400:90 }
2024-01-27 14:56:21,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-27 14:56:21,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv975343400=29, srv1473292856=11, srv1125976871=2, srv1592347826=14, srv1193261949=7, srv1857802255=19, srv1896180909=20, srv3203305=24, srv1749195524=16, srv1430096690=9, srv1718027539=15, srv1475594564=12, srv307889131=23, srv115675271=4, srv1532251581=13, srv560236243=26, srv112804577=3, srv1195616860=8, srv1438141222=10, srv1821492877=18, srv435193899=25, srv1015722041=1, srv225109012=22, srv2131922745=21, srv7106960=28, srv1171160567=5, srv612141355=27, srv1192374035=6, srv1793361803=17, srv1002329352=0} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-27 14:56:21,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-27 14:56:21,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-27 14:56:21,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-27 14:56:21,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-27 14:56:21,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-27 14:56:21,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-27 14:56:21,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-27 14:56:21,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-27 14:56:21,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-27 14:56:21,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-27 14:56:21,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-27 14:56:21,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-27 14:56:21,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-27 14:56:21,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-27 14:56:21,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-27 14:56:21,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-27 14:56:21,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-27 14:56:21,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-27 14:56:21,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-27 14:56:21,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-27 14:56:21,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-27 14:56:21,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-27 14:56:21,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-27 14:56:21,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-27 14:56:21,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-27 14:56:21,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-27 14:56:21,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-27 14:56:21,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-27 14:56:21,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-27 14:56:21,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-27 14:56:21,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-01-27 14:56:21,972 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-27 14:56:21,972 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1002329352:90 , srv1015722041:90 , srv1125976871:90 , srv112804577:90 , srv115675271:90 , srv1171160567:90 , srv1192374035:90 , srv1193261949:90 , srv1195616860:90 , srv1430096690:90 , srv1438141222:90 , srv1473292856:90 , srv1475594564:90 , srv1532251581:90 , srv1592347826:90 , srv1718027539:90 , srv1749195524:90 , srv1793361803:90 , srv1821492877:90 , srv1857802255:90 , srv1896180909:90 , srv2131922745:90 , srv225109012:90 , srv307889131:90 , srv3203305:90 , srv435193899:90 , srv560236243:90 , srv612141355:90 , srv7106960:90 , srv975343400:90 }
2024-01-27 14:56:21,983 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=136 (was 257), ProcessCount=174 (was 176), AvailableMemoryMB=2857 (was 1683) - AvailableMemoryMB LEAK? -