2023-12-22 15:53:19,181 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-12-22 15:53:19,518 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-12-22 15:53:19,638 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-12-22 15:53:19,711 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-12-22 15:53:19,720 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-12-22 15:53:19,734 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=277, ProcessCount=169, AvailableMemoryMB=4255
2023-12-22 15:53:19,736 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-12-22 15:53:19,736 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-12-22 15:53:19,860 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1000982526:84 , srv1018263032:84 , srv1042128016:84 , srv1046161186:84 , srv1057120007:84 , srv1080210242:84 , srv1091048372:84 , srv1093142466:84 , srv1103115728:84 , srv1106360083:84 , srv1110790118:84 , srv1207675652:84 , srv121158979:84 , srv122274139:84 , srv1243748654:84 , srv1250452677:84 , srv125886033:84 , srv1259828606:84 , srv1281216523:84 , srv1284759907:84 , srv1312738254:84 , srv1341927347:84 , srv1371441593:84 , srv1373208727:84 , srv138427907:84 , srv1398261285:84 , srv1409934303:84 , srv1423495078:84 , srv1434278421:84 , srv1539773508:84 , srv1560509146:84 , srv1585041907:84 , srv1587408476:84 , srv1631515694:84 , srv1631893099:84 , srv1681284502:84 , srv1728630084:84 , srv1730545768:84 , srv1789276299:84 , srv1831458807:84 , srv1832053526:84 , srv1832899467:84 , srv1835564010:84 , srv1846571167:84 , srv1846904733:84 , srv184776835:84 , srv1856446740:84 , srv1866947714:84 , srv1870201737:84 , srv1871862755:84 , srv1872141560:84 , srv1893353846:84 , srv1903686508:84 , srv1916456264:84 , srv1928951301:84 , srv1971885478:84 , srv197445714:84 , srv2019463567:84 , srv2092946680:84 , srv2097272389:84 , srv2120619580:84 , srv2138900488:84 , srv237676514:84 , srv243121308:84 , srv260947998:84 , srv278451447:84 , srv286671643:84 , srv331369194:84 , srv341350401:84 , srv369761553:84 , srv390552721:84 , srv399277216:84 , srv399416789:84 , srv461024961:84 , srv461285944:84 , srv466241013:84 , srv488099850:84 , srv494677777:84 , srv519176840:84 , srv524717150:84 , srv56530639:84 , srv576172049:84 , srv581144574:84 , srv625397607:84 , srv630712624:84 , srv632427625:84 , srv635741650:84 , srv660757468:84 , srv670799839:84 , srv700534246:84 , srv711880969:84 , srv722764461:84 , srv810824157:84 , srv903846884:84 , srv908433196:84 , srv937966858:84 , srv947751917:84 , srv985672733:84 , srv986182767:84 , srv1135233488:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-12-22 15:53:19,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-22 15:53:19,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv184776835=46, srv369761553=70, srv1832053526=41, srv1866947714=48, srv1832899467=42, srv466241013=76, srv1871862755=50, srv461024961=74, srv1789276299=39, srv1872141560=51, srv1916456264=54, srv1243748654=15, srv390552721=71, srv331369194=68, srv1560509146=31, srv122274139=14, srv2097272389=60, srv1371441593=23, srv632427625=86, srv1312738254=21, srv903846884=94, srv243121308=64, srv125886033=17, srv670799839=89, srv1135233488=11, srv1831458807=40, srv986182767=99, srv1046161186=3, srv1631515694=34, srv947751917=97, srv1042128016=2, srv1259828606=18, srv494677777=78, srv937966858=96, srv1631893099=35, srv1730545768=38, srv2120619580=61, srv576172049=82, srv1893353846=52, srv461285944=75, srv625397607=84, srv2019463567=58, srv278451447=66, srv1409934303=27, srv630712624=85, srv488099850=77, srv1835564010=43, srv1250452677=16, srv1434278421=29, srv1373208727=24, srv237676514=63, srv581144574=83, srv1207675652=12, srv635741650=87, srv197445714=57, srv524717150=80, srv1000982526=0, srv1971885478=56, srv1846571167=44, srv121158979=13, srv1103115728=8, srv1681284502=36, srv1846904733=45, srv1398261285=26, srv1281216523=19, srv1585041907=32, srv1928951301=55, srv260947998=65, srv399416789=73, srv341350401=69, srv1423495078=28, srv399277216=72, srv1870201737=49, srv2138900488=62, srv1539773508=30, srv1587408476=33, srv1080210242=5, srv138427907=25, srv1093142466=7, srv2092946680=59, srv286671643=67, srv700534246=90, srv1091048372=6, srv1106360083=9, srv711880969=91, srv908433196=95, srv1728630084=37, srv1018263032=1, srv1110790118=10, srv56530639=81, srv985672733=98, srv722764461=92, srv1284759907=20, srv519176840=79, srv660757468=88, srv1057120007=4, srv1341927347=22, srv1856446740=47, srv1903686508=53, srv810824157=93} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-22 15:53:19,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-22 15:53:19,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-22 15:53:19,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-22 15:53:19,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-22 15:53:19,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-22 15:53:19,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-22 15:53:19,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-22 15:53:19,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-22 15:53:19,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-22 15:53:19,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-22 15:53:19,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-22 15:53:19,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-22 15:53:19,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-22 15:53:19,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-22 15:53:19,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-22 15:53:19,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-22 15:53:19,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-22 15:53:19,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-22 15:53:19,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-22 15:53:19,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-22 15:53:19,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-22 15:53:19,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-22 15:53:19,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-22 15:53:19,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-22 15:53:19,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-22 15:53:19,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-22 15:53:19,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-22 15:53:19,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-22 15:53:19,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-22 15:53:19,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-22 15:53:19,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-22 15:53:19,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-22 15:53:19,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-22 15:53:19,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-22 15:53:19,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-22 15:53:19,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-22 15:53:19,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-22 15:53:19,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-22 15:53:19,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-22 15:53:19,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-22 15:53:19,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-22 15:53:19,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-22 15:53:19,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-22 15:53:19,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-22 15:53:19,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-22 15:53:19,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-22 15:53:19,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-22 15:53:19,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-22 15:53:19,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-22 15:53:19,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-22 15:53:19,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-22 15:53:19,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-22 15:53:19,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-22 15:53:19,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-22 15:53:19,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-22 15:53:19,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-22 15:53:19,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-22 15:53:19,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-22 15:53:19,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-22 15:53:19,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-22 15:53:19,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-22 15:53:19,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-22 15:53:19,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-22 15:53:19,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-22 15:53:19,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-22 15:53:19,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-22 15:53:19,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-22 15:53:19,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-22 15:53:19,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-22 15:53:19,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-22 15:53:19,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-22 15:53:19,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-22 15:53:19,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-22 15:53:19,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-22 15:53:19,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-22 15:53:19,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-22 15:53:19,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-22 15:53:19,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-22 15:53:19,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-22 15:53:19,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-22 15:53:19,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-22 15:53:19,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-22 15:53:19,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-22 15:53:19,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-22 15:53:19,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-22 15:53:19,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-22 15:53:19,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-22 15:53:19,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-22 15:53:19,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-22 15:53:19,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-22 15:53:19,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-22 15:53:19,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-22 15:53:19,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-22 15:53:19,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-22 15:53:19,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-22 15:53:19,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-22 15:53:19,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-22 15:53:19,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-22 15:53:19,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-22 15:53:19,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-22 15:53:20,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-22 15:53:20,228 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-22 15:53:20,228 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-12-22 15:53:20,229 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-12-22 15:53:20,614 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 577 ms to try 5000 different iterations.  Found a solution that moves 4671 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17890645659464383. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040619040758352404, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.057389918048405444, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.519, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040619040758352404, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17425, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23933333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-22 15:53:20,639 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv722764461:81 , srv1585041907:82 , srv1728630084:82 , srv625397607:82 , srv1000982526:83 , srv1250452677:83 , srv1371441593:83 , srv138427907:83 , srv1423495078:83 , srv1831458807:83 , srv184776835:83 , srv1903686508:83 , srv399277216:83 , srv461024961:83 , srv488099850:83 , srv494677777:83 , srv700534246:83 , srv1057120007:84 , srv1284759907:84 , srv1373208727:84 , srv1856446740:84 , srv1971885478:84 , srv2138900488:84 , srv243121308:84 , srv576172049:84 , srv1091048372:85 , srv1832053526:85 , srv1832899467:85 , srv1846904733:85 , srv1893353846:85 , srv1916456264:85 , srv341350401:85 , srv466241013:85 , srv581144574:85 , srv630712624:85 , srv635741650:85 , srv986182767:85 , srv1042128016:86 , srv1080210242:86 , srv1106360083:86 , srv1110790118:86 , srv1243748654:86 , srv1312738254:86 , srv1341927347:86 , srv2097272389:86 , srv2120619580:86 , srv237676514:86 , srv286671643:86 , srv331369194:86 , srv56530639:86 , srv632427625:86 , srv670799839:86 , srv908433196:86 , srv937966858:86 , srv1018263032:87 , srv1259828606:87 , srv1281216523:87 , srv1631893099:87 , srv1789276299:87 , srv1866947714:87 , srv1872141560:87 , srv278451447:87 , srv369761553:87 , srv399416789:87 , srv519176840:87 , srv660757468:87 , srv903846884:87 , srv985672733:87 , srv1093142466:88 , srv1207675652:88 , srv122274139:88 , srv125886033:88 , srv1398261285:88 , srv1434278421:88 , srv1539773508:88 , srv1560509146:88 , srv1587408476:88 , srv2019463567:88 , srv524717150:88 , srv711880969:88 , srv810824157:88 , srv1103115728:89 , srv1409934303:89 , srv1631515694:89 , srv1846571167:89 , srv197445714:89 , srv461285944:89 , srv1046161186:90 , srv1681284502:90 , srv1871862755:90 , srv2092946680:90 , srv390552721:90 , srv947751917:90 , srv1835564010:91 , srv121158979:92 , srv1870201737:92 , srv1928951301:92 , srv1730545768:93 , srv260947998:93 , srv1135233488:451 }
2023-12-22 15:53:20,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-22 15:53:20,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv184776835=46, srv369761553=70, srv1832053526=41, srv1866947714=48, srv1832899467=42, srv466241013=76, srv1871862755=50, srv461024961=74, srv1789276299=39, srv1872141560=51, srv1916456264=54, srv1243748654=15, srv390552721=71, srv331369194=68, srv1560509146=31, srv122274139=14, srv2097272389=60, srv1371441593=23, srv632427625=86, srv1312738254=21, srv903846884=94, srv243121308=64, srv125886033=17, srv670799839=89, srv1135233488=11, srv1831458807=40, srv986182767=99, srv1046161186=3, srv1631515694=34, srv947751917=97, srv1042128016=2, srv1259828606=18, srv494677777=78, srv937966858=96, srv1631893099=35, srv1730545768=38, srv2120619580=61, srv576172049=82, srv1893353846=52, srv461285944=75, srv625397607=84, srv2019463567=58, srv278451447=66, srv1409934303=27, srv630712624=85, srv488099850=77, srv1835564010=43, srv1250452677=16, srv1434278421=29, srv1373208727=24, srv237676514=63, srv581144574=83, srv1207675652=12, srv635741650=87, srv197445714=57, srv524717150=80, srv1000982526=0, srv1971885478=56, srv1846571167=44, srv121158979=13, srv1103115728=8, srv1681284502=36, srv1846904733=45, srv1398261285=26, srv1281216523=19, srv1585041907=32, srv1928951301=55, srv260947998=65, srv399416789=73, srv341350401=69, srv1423495078=28, srv399277216=72, srv1870201737=49, srv2138900488=62, srv1539773508=30, srv1587408476=33, srv1080210242=5, srv138427907=25, srv1093142466=7, srv2092946680=59, srv286671643=67, srv700534246=90, srv1091048372=6, srv1106360083=9, srv711880969=91, srv908433196=95, srv1728630084=37, srv1018263032=1, srv1110790118=10, srv56530639=81, srv985672733=98, srv722764461=92, srv1284759907=20, srv519176840=79, srv660757468=88, srv1057120007=4, srv1341927347=22, srv1856446740=47, srv1903686508=53, srv810824157=93} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-22 15:53:20,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-22 15:53:20,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-22 15:53:20,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-22 15:53:20,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-22 15:53:20,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-22 15:53:20,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-22 15:53:20,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-22 15:53:20,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-22 15:53:20,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-22 15:53:20,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-22 15:53:20,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-22 15:53:20,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-22 15:53:20,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-22 15:53:20,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-22 15:53:20,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-22 15:53:20,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-22 15:53:20,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-22 15:53:20,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-22 15:53:20,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-22 15:53:20,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-22 15:53:20,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-22 15:53:20,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-22 15:53:20,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-22 15:53:20,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-22 15:53:20,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-22 15:53:20,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-22 15:53:20,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-22 15:53:20,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-22 15:53:20,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-22 15:53:20,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-22 15:53:20,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-22 15:53:20,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-22 15:53:20,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-22 15:53:20,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-22 15:53:20,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-22 15:53:20,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-22 15:53:20,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-22 15:53:20,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-22 15:53:20,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-22 15:53:20,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-22 15:53:20,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-22 15:53:20,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-22 15:53:20,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-22 15:53:20,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-22 15:53:20,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-22 15:53:20,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-22 15:53:20,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-22 15:53:20,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-22 15:53:20,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-22 15:53:20,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-22 15:53:20,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-22 15:53:20,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-22 15:53:20,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-22 15:53:20,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-22 15:53:20,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-22 15:53:20,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-22 15:53:20,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-22 15:53:20,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-22 15:53:20,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-22 15:53:20,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-22 15:53:20,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-22 15:53:20,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-22 15:53:20,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-22 15:53:20,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-22 15:53:20,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-22 15:53:20,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-22 15:53:20,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-22 15:53:20,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-22 15:53:20,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-22 15:53:20,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-22 15:53:20,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-22 15:53:20,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-22 15:53:20,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-22 15:53:20,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-22 15:53:20,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-22 15:53:20,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-22 15:53:20,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-22 15:53:20,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-22 15:53:20,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-22 15:53:20,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-22 15:53:20,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-22 15:53:20,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-22 15:53:20,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-22 15:53:20,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-22 15:53:20,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-22 15:53:20,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-22 15:53:20,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-22 15:53:20,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-22 15:53:20,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-22 15:53:20,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-22 15:53:20,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-22 15:53:20,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-22 15:53:20,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-22 15:53:20,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-22 15:53:20,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-22 15:53:20,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-22 15:53:20,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-22 15:53:20,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-22 15:53:20,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-22 15:53:20,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-22 15:53:20,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-22 15:53:20,811 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040619040758352404, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.057389918048405444, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040619040758352404, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17425, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23933333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-22 15:53:20,811 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-12-22 15:53:20,811 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17887374955358223, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040619040758352404, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.057389918048405444, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040619040758352404, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17425, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23933333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-12-22 15:53:21,094 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 396 ms to try 5000 different iterations.  Found a solution that moves 2629 regions; Going from a computed imbalance of 0.17887374955358223 to a new imbalance of 0.015262337262924976. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020919146126161834); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03244956373061924, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2921111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020919146126161834); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01225); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.044083333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-22 15:53:21,103 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv260947998:79 , srv121158979:80 , srv1835564010:80 , srv1870201737:80 , srv1046161186:81 , srv1730545768:81 , srv1928951301:81 , srv1560509146:82 , srv1871862755:82 , srv197445714:82 , srv390552721:82 , srv461285944:82 , srv1093142466:83 , srv1434278421:83 , srv1631515694:83 , srv1681284502:83 , srv1846571167:83 , srv2092946680:83 , srv1042128016:84 , srv1103115728:84 , srv1409934303:84 , srv1539773508:84 , srv1631893099:84 , srv1866947714:84 , srv2019463567:84 , srv278451447:84 , srv466241013:84 , srv632427625:84 , srv711880969:84 , srv810824157:84 , srv1018263032:85 , srv1091048372:85 , srv1207675652:85 , srv125886033:85 , srv1259828606:85 , srv1587408476:85 , srv2097272389:85 , srv2120619580:85 , srv369761553:85 , srv524717150:85 , srv1080210242:86 , srv1106360083:86 , srv1110790118:86 , srv122274139:86 , srv1243748654:86 , srv1281216523:86 , srv1341927347:86 , srv1373208727:86 , srv1398261285:86 , srv1832899467:86 , srv1872141560:86 , srv237676514:86 , srv286671643:86 , srv581144574:86 , srv635741650:86 , srv660757468:86 , srv903846884:86 , srv908433196:86 , srv937966858:86 , srv947751917:86 , srv986182767:86 , srv1312738254:87 , srv1789276299:87 , srv1856446740:87 , srv1893353846:87 , srv1971885478:87 , srv2138900488:87 , srv243121308:87 , srv331369194:87 , srv341350401:87 , srv399416789:87 , srv56530639:87 , srv630712624:87 , srv985672733:87 , srv1250452677:88 , srv1423495078:88 , srv1728630084:88 , srv1832053526:88 , srv1846904733:88 , srv1916456264:88 , srv494677777:88 , srv519176840:88 , srv700534246:88 , srv1284759907:89 , srv1831458807:89 , srv184776835:89 , srv1903686508:89 , srv399277216:89 , srv461024961:89 , srv670799839:89 , srv1000982526:90 , srv1057120007:90 , srv1371441593:90 , srv138427907:90 , srv1585041907:90 , srv488099850:90 , srv576172049:90 , srv722764461:90 , srv625397607:91 , srv1135233488:509 }
2023-12-22 15:53:21,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-22 15:53:21,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv184776835=46, srv369761553=70, srv1832053526=41, srv1866947714=48, srv1832899467=42, srv466241013=76, srv1871862755=50, srv461024961=74, srv1789276299=39, srv1872141560=51, srv1916456264=54, srv1243748654=15, srv390552721=71, srv331369194=68, srv1560509146=31, srv122274139=14, srv2097272389=60, srv1371441593=23, srv632427625=86, srv1312738254=21, srv903846884=94, srv243121308=64, srv125886033=17, srv670799839=89, srv1135233488=11, srv1831458807=40, srv986182767=99, srv1046161186=3, srv1631515694=34, srv947751917=97, srv1042128016=2, srv1259828606=18, srv494677777=78, srv937966858=96, srv1631893099=35, srv1730545768=38, srv2120619580=61, srv576172049=82, srv1893353846=52, srv461285944=75, srv625397607=84, srv2019463567=58, srv278451447=66, srv1409934303=27, srv630712624=85, srv488099850=77, srv1835564010=43, srv1250452677=16, srv1434278421=29, srv1373208727=24, srv237676514=63, srv581144574=83, srv1207675652=12, srv635741650=87, srv197445714=57, srv524717150=80, srv1000982526=0, srv1971885478=56, srv1846571167=44, srv121158979=13, srv1103115728=8, srv1681284502=36, srv1846904733=45, srv1398261285=26, srv1281216523=19, srv1585041907=32, srv1928951301=55, srv260947998=65, srv399416789=73, srv341350401=69, srv1423495078=28, srv399277216=72, srv1870201737=49, srv2138900488=62, srv1539773508=30, srv1587408476=33, srv1080210242=5, srv138427907=25, srv1093142466=7, srv2092946680=59, srv286671643=67, srv700534246=90, srv1091048372=6, srv1106360083=9, srv711880969=91, srv908433196=95, srv1728630084=37, srv1018263032=1, srv1110790118=10, srv56530639=81, srv985672733=98, srv722764461=92, srv1284759907=20, srv519176840=79, srv660757468=88, srv1057120007=4, srv1341927347=22, srv1856446740=47, srv1903686508=53, srv810824157=93} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-22 15:53:21,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-22 15:53:21,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-22 15:53:21,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-22 15:53:21,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-22 15:53:21,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-22 15:53:21,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-22 15:53:21,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-22 15:53:21,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-22 15:53:21,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-22 15:53:21,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-22 15:53:21,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-22 15:53:21,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-22 15:53:21,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-22 15:53:21,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-22 15:53:21,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-22 15:53:21,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-22 15:53:21,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-22 15:53:21,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-22 15:53:21,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-22 15:53:21,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-22 15:53:21,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-22 15:53:21,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-22 15:53:21,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-22 15:53:21,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-22 15:53:21,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-22 15:53:21,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-22 15:53:21,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-22 15:53:21,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-22 15:53:21,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-22 15:53:21,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-22 15:53:21,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-22 15:53:21,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-22 15:53:21,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-22 15:53:21,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-22 15:53:21,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-22 15:53:21,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-22 15:53:21,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-22 15:53:21,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-22 15:53:21,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-22 15:53:21,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-22 15:53:21,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-22 15:53:21,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-22 15:53:21,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-22 15:53:21,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-22 15:53:21,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-22 15:53:21,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-22 15:53:21,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-22 15:53:21,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-22 15:53:21,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-22 15:53:21,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-22 15:53:21,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-22 15:53:21,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-22 15:53:21,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-22 15:53:21,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-22 15:53:21,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-22 15:53:21,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-22 15:53:21,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-22 15:53:21,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-22 15:53:21,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-22 15:53:21,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-22 15:53:21,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-22 15:53:21,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-22 15:53:21,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-22 15:53:21,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-22 15:53:21,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-22 15:53:21,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-22 15:53:21,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-22 15:53:21,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-22 15:53:21,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-22 15:53:21,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-22 15:53:21,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-22 15:53:21,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-22 15:53:21,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-22 15:53:21,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-22 15:53:21,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-22 15:53:21,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-22 15:53:21,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-22 15:53:21,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-22 15:53:21,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-22 15:53:21,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-22 15:53:21,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-22 15:53:21,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-22 15:53:21,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-22 15:53:21,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-22 15:53:21,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-22 15:53:21,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-22 15:53:21,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-22 15:53:21,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-22 15:53:21,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-22 15:53:21,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-22 15:53:21,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-22 15:53:21,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-22 15:53:21,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-22 15:53:21,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-22 15:53:21,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-22 15:53:21,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-22 15:53:21,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-22 15:53:21,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-22 15:53:21,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-22 15:53:21,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-22 15:53:21,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-22 15:53:21,274 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020919146126161834); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03244956373061924, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020919146126161834); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01225); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.044083333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-22 15:53:21,274 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-12-22 15:53:21,274 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.015243928611468975, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020919146126161834); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03244956373061924, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020919146126161834); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01225); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.044083333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-12-22 15:53:21,512 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 343 ms to try 5000 different iterations.  Found a solution that moves 992 regions; Going from a computed imbalance of 0.015243928611468975 to a new imbalance of 5.5015319759995E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.003548955715300029); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005211573066470477); MoveCostFunction : (multiplier=7.0, imbalance=0.11022222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.003548955715300029); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0030833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-22 15:53:21,515 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv519176840:82 , srv1789276299:83 , srv670799839:83 , srv947751917:83 , srv1057120007:84 , srv1207675652:84 , srv122274139:84 , srv1281216523:84 , srv1312738254:84 , srv1371441593:84 , srv1409934303:84 , srv1681284502:84 , srv1730545768:84 , srv1872141560:84 , srv399416789:84 , srv576172049:84 , srv660757468:84 , srv985672733:84 , srv1000982526:85 , srv1103115728:85 , srv1106360083:85 , srv121158979:85 , srv125886033:85 , srv1341927347:85 , srv138427907:85 , srv1398261285:85 , srv1587408476:85 , srv1832053526:85 , srv1846904733:85 , srv184776835:85 , srv1871862755:85 , srv1893353846:85 , srv1916456264:85 , srv2092946680:85 , srv237676514:85 , srv286671643:85 , srv331369194:85 , srv341350401:85 , srv488099850:85 , srv524717150:85 , srv56530639:85 , srv625397607:85 , srv711880969:85 , srv903846884:85 , srv908433196:85 , srv937966858:85 , srv1018263032:86 , srv1080210242:86 , srv1093142466:86 , srv1110790118:86 , srv1243748654:86 , srv1250452677:86 , srv1259828606:86 , srv1284759907:86 , srv1434278421:86 , srv1585041907:86 , srv1631515694:86 , srv1631893099:86 , srv1831458807:86 , srv1835564010:86 , srv1846571167:86 , srv1856446740:86 , srv1870201737:86 , srv1903686508:86 , srv1928951301:86 , srv197445714:86 , srv2019463567:86 , srv2120619580:86 , srv260947998:86 , srv369761553:86 , srv390552721:86 , srv399277216:86 , srv461024961:86 , srv494677777:86 , srv581144574:86 , srv630712624:86 , srv632427625:86 , srv635741650:86 , srv810824157:86 , srv1046161186:87 , srv1373208727:87 , srv1423495078:87 , srv1539773508:87 , srv1560509146:87 , srv1728630084:87 , srv1832899467:87 , srv1866947714:87 , srv1971885478:87 , srv2097272389:87 , srv243121308:87 , srv278451447:87 , srv461285944:87 , srv700534246:87 , srv722764461:87 , srv986182767:87 , srv1042128016:88 , srv1091048372:88 , srv2138900488:89 , srv466241013:89 , srv1135233488:529 }
2023-12-22 15:53:21,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-22 15:53:21,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv184776835=46, srv369761553=70, srv1832053526=41, srv1866947714=48, srv1832899467=42, srv466241013=76, srv1871862755=50, srv461024961=74, srv1789276299=39, srv1872141560=51, srv1916456264=54, srv1243748654=15, srv390552721=71, srv331369194=68, srv1560509146=31, srv122274139=14, srv2097272389=60, srv1371441593=23, srv632427625=86, srv1312738254=21, srv903846884=94, srv243121308=64, srv125886033=17, srv670799839=89, srv1135233488=11, srv1831458807=40, srv986182767=99, srv1046161186=3, srv1631515694=34, srv947751917=97, srv1042128016=2, srv1259828606=18, srv494677777=78, srv937966858=96, srv1631893099=35, srv1730545768=38, srv2120619580=61, srv576172049=82, srv1893353846=52, srv461285944=75, srv625397607=84, srv2019463567=58, srv278451447=66, srv1409934303=27, srv630712624=85, srv488099850=77, srv1835564010=43, srv1250452677=16, srv1434278421=29, srv1373208727=24, srv237676514=63, srv581144574=83, srv1207675652=12, srv635741650=87, srv197445714=57, srv524717150=80, srv1000982526=0, srv1971885478=56, srv1846571167=44, srv121158979=13, srv1103115728=8, srv1681284502=36, srv1846904733=45, srv1398261285=26, srv1281216523=19, srv1585041907=32, srv1928951301=55, srv260947998=65, srv399416789=73, srv341350401=69, srv1423495078=28, srv399277216=72, srv1870201737=49, srv2138900488=62, srv1539773508=30, srv1587408476=33, srv1080210242=5, srv138427907=25, srv1093142466=7, srv2092946680=59, srv286671643=67, srv700534246=90, srv1091048372=6, srv1106360083=9, srv711880969=91, srv908433196=95, srv1728630084=37, srv1018263032=1, srv1110790118=10, srv56530639=81, srv985672733=98, srv722764461=92, srv1284759907=20, srv519176840=79, srv660757468=88, srv1057120007=4, srv1341927347=22, srv1856446740=47, srv1903686508=53, srv810824157=93} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-22 15:53:21,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-22 15:53:21,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-22 15:53:21,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-22 15:53:21,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-22 15:53:21,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-22 15:53:21,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-22 15:53:21,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-22 15:53:21,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-22 15:53:21,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-22 15:53:21,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-22 15:53:21,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-22 15:53:21,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-22 15:53:21,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-22 15:53:21,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-22 15:53:21,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-22 15:53:21,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-22 15:53:21,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-22 15:53:21,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-22 15:53:21,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-22 15:53:21,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-22 15:53:21,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-22 15:53:21,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-22 15:53:21,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-22 15:53:21,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-22 15:53:21,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-22 15:53:21,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-22 15:53:21,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-22 15:53:21,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-22 15:53:21,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-22 15:53:21,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-22 15:53:21,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-22 15:53:21,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-22 15:53:21,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-22 15:53:21,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-22 15:53:21,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-22 15:53:21,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-22 15:53:21,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-22 15:53:21,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-22 15:53:21,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-22 15:53:21,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-22 15:53:21,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-22 15:53:21,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-22 15:53:21,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-22 15:53:21,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-22 15:53:21,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-22 15:53:21,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-22 15:53:21,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-22 15:53:21,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-22 15:53:21,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-22 15:53:21,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-22 15:53:21,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-22 15:53:21,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-22 15:53:21,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-22 15:53:21,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-22 15:53:21,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-22 15:53:21,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-22 15:53:21,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-22 15:53:21,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-22 15:53:21,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-22 15:53:21,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-22 15:53:21,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-22 15:53:21,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-22 15:53:21,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-22 15:53:21,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-22 15:53:21,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-22 15:53:21,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-22 15:53:21,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-22 15:53:21,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-22 15:53:21,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-22 15:53:21,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-22 15:53:21,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-22 15:53:21,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-22 15:53:21,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-22 15:53:21,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-22 15:53:21,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-22 15:53:21,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-22 15:53:21,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-22 15:53:21,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-22 15:53:21,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-22 15:53:21,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-22 15:53:21,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-22 15:53:21,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-22 15:53:21,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-22 15:53:21,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-22 15:53:21,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-22 15:53:21,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-22 15:53:21,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-22 15:53:21,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-22 15:53:21,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-22 15:53:21,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-22 15:53:21,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-22 15:53:21,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-22 15:53:21,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-22 15:53:21,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-22 15:53:21,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-22 15:53:21,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-22 15:53:21,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-22 15:53:21,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-22 15:53:21,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-22 15:53:21,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-22 15:53:21,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-22 15:53:21,659 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.003548955715300029); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005211573066470477); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.003548955715300029); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0030833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-22 15:53:21,659 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-12-22 15:53:21,660 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=5.432070651372839E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.003548955715300029); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005211573066470477); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.003548955715300029); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0030833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-12-22 15:53:21,998 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 445 ms to try 5000 different iterations.  Found a solution that moves 169 regions; Going from a computed imbalance of 5.432070651372839E-4 to a new imbalance of 9.448731161769105E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.158529131194229E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014982219165849825); MoveCostFunction : (multiplier=7.0, imbalance=0.01877777777777778); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.158529131194229E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-22 15:53:22,000 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv2138900488:82 , srv1284759907:83 , srv1398261285:83 , srv1539773508:83 , srv1928951301:83 , srv947751917:83 , srv1000982526:84 , srv1018263032:84 , srv1042128016:84 , srv1046161186:84 , srv1057120007:84 , srv1080210242:84 , srv1091048372:84 , srv1103115728:84 , srv121158979:84 , srv122274139:84 , srv1243748654:84 , srv125886033:84 , srv1259828606:84 , srv138427907:84 , srv1423495078:84 , srv1585041907:84 , srv1587408476:84 , srv1631515694:84 , srv1681284502:84 , srv1730545768:84 , srv1831458807:84 , srv1832053526:84 , srv1832899467:84 , srv1846571167:84 , srv1846904733:84 , srv1856446740:84 , srv1866947714:84 , srv1870201737:84 , srv1872141560:84 , srv1903686508:84 , srv1916456264:84 , srv1971885478:84 , srv2019463567:84 , srv2092946680:84 , srv2097272389:84 , srv243121308:84 , srv260947998:84 , srv278451447:84 , srv331369194:84 , srv369761553:84 , srv390552721:84 , srv399416789:84 , srv461024961:84 , srv461285944:84 , srv466241013:84 , srv488099850:84 , srv524717150:84 , srv56530639:84 , srv576172049:84 , srv625397607:84 , srv630712624:84 , srv700534246:84 , srv722764461:84 , srv810824157:84 , srv903846884:84 , srv986182767:84 , srv1093142466:85 , srv1106360083:85 , srv1110790118:85 , srv1207675652:85 , srv1250452677:85 , srv1281216523:85 , srv1312738254:85 , srv1341927347:85 , srv1371441593:85 , srv1373208727:85 , srv1409934303:85 , srv1434278421:85 , srv1560509146:85 , srv1631893099:85 , srv1728630084:85 , srv1789276299:85 , srv1835564010:85 , srv184776835:85 , srv1871862755:85 , srv1893353846:85 , srv197445714:85 , srv2120619580:85 , srv237676514:85 , srv286671643:85 , srv341350401:85 , srv399277216:85 , srv494677777:85 , srv519176840:85 , srv581144574:85 , srv635741650:85 , srv660757468:85 , srv670799839:85 , srv711880969:85 , srv908433196:85 , srv937966858:85 , srv985672733:85 , srv632427625:86 , srv1135233488:653 }
2023-12-22 15:53:22,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-22 15:53:22,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv184776835=46, srv369761553=70, srv1832053526=41, srv1866947714=48, srv1832899467=42, srv466241013=76, srv1871862755=50, srv461024961=74, srv1789276299=39, srv1872141560=51, srv1916456264=54, srv1243748654=15, srv390552721=71, srv331369194=68, srv1560509146=31, srv122274139=14, srv2097272389=60, srv1371441593=23, srv632427625=86, srv1312738254=21, srv903846884=94, srv243121308=64, srv125886033=17, srv670799839=89, srv1135233488=11, srv1831458807=40, srv986182767=99, srv1046161186=3, srv1631515694=34, srv947751917=97, srv1042128016=2, srv1259828606=18, srv494677777=78, srv937966858=96, srv1631893099=35, srv1730545768=38, srv2120619580=61, srv576172049=82, srv1893353846=52, srv461285944=75, srv625397607=84, srv2019463567=58, srv278451447=66, srv1409934303=27, srv630712624=85, srv488099850=77, srv1835564010=43, srv1250452677=16, srv1434278421=29, srv1373208727=24, srv237676514=63, srv581144574=83, srv1207675652=12, srv635741650=87, srv197445714=57, srv524717150=80, srv1000982526=0, srv1971885478=56, srv1846571167=44, srv121158979=13, srv1103115728=8, srv1681284502=36, srv1846904733=45, srv1398261285=26, srv1281216523=19, srv1585041907=32, srv1928951301=55, srv260947998=65, srv399416789=73, srv341350401=69, srv1423495078=28, srv399277216=72, srv1870201737=49, srv2138900488=62, srv1539773508=30, srv1587408476=33, srv1080210242=5, srv138427907=25, srv1093142466=7, srv2092946680=59, srv286671643=67, srv700534246=90, srv1091048372=6, srv1106360083=9, srv711880969=91, srv908433196=95, srv1728630084=37, srv1018263032=1, srv1110790118=10, srv56530639=81, srv985672733=98, srv722764461=92, srv1284759907=20, srv519176840=79, srv660757468=88, srv1057120007=4, srv1341927347=22, srv1856446740=47, srv1903686508=53, srv810824157=93} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-22 15:53:22,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-22 15:53:22,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-22 15:53:22,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-22 15:53:22,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-22 15:53:22,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-22 15:53:22,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-22 15:53:22,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-22 15:53:22,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-22 15:53:22,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-22 15:53:22,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-22 15:53:22,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-22 15:53:22,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-22 15:53:22,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-22 15:53:22,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-22 15:53:22,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-22 15:53:22,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-22 15:53:22,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-22 15:53:22,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-22 15:53:22,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-22 15:53:22,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-22 15:53:22,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-22 15:53:22,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-22 15:53:22,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-22 15:53:22,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-22 15:53:22,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-22 15:53:22,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-22 15:53:22,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-22 15:53:22,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-22 15:53:22,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-22 15:53:22,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-22 15:53:22,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-22 15:53:22,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-22 15:53:22,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-22 15:53:22,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-22 15:53:22,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-22 15:53:22,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-22 15:53:22,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-22 15:53:22,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-22 15:53:22,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-22 15:53:22,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-22 15:53:22,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-22 15:53:22,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-22 15:53:22,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-22 15:53:22,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-22 15:53:22,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-22 15:53:22,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-22 15:53:22,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-22 15:53:22,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-22 15:53:22,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-22 15:53:22,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-22 15:53:22,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-22 15:53:22,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-22 15:53:22,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-22 15:53:22,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-22 15:53:22,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-22 15:53:22,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-22 15:53:22,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-22 15:53:22,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-22 15:53:22,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-22 15:53:22,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-22 15:53:22,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-22 15:53:22,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-22 15:53:22,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-22 15:53:22,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-22 15:53:22,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-22 15:53:22,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-22 15:53:22,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-22 15:53:22,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-22 15:53:22,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-22 15:53:22,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-22 15:53:22,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-22 15:53:22,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-22 15:53:22,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-22 15:53:22,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-22 15:53:22,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-22 15:53:22,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-22 15:53:22,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-22 15:53:22,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-22 15:53:22,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-22 15:53:22,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-22 15:53:22,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-22 15:53:22,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-22 15:53:22,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-22 15:53:22,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-22 15:53:22,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-22 15:53:22,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-22 15:53:22,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-22 15:53:22,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-22 15:53:22,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-22 15:53:22,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-22 15:53:22,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-22 15:53:22,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-22 15:53:22,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-22 15:53:22,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-22 15:53:22,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-22 15:53:22,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-22 15:53:22,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-22 15:53:22,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-22 15:53:22,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-22 15:53:22,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-22 15:53:22,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-22 15:53:22,132 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=8.265367869238297E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.158529131194229E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014982219165849825); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.158529131194229E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-22 15:53:22,132 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv2138900488:82 , srv1284759907:83 , srv1398261285:83 , srv1539773508:83 , srv1928951301:83 , srv947751917:83 , srv1000982526:84 , srv1018263032:84 , srv1042128016:84 , srv1046161186:84 , srv1057120007:84 , srv1080210242:84 , srv1091048372:84 , srv1103115728:84 , srv121158979:84 , srv122274139:84 , srv1243748654:84 , srv125886033:84 , srv1259828606:84 , srv138427907:84 , srv1423495078:84 , srv1585041907:84 , srv1587408476:84 , srv1631515694:84 , srv1681284502:84 , srv1730545768:84 , srv1831458807:84 , srv1832053526:84 , srv1832899467:84 , srv1846571167:84 , srv1846904733:84 , srv1856446740:84 , srv1866947714:84 , srv1870201737:84 , srv1872141560:84 , srv1903686508:84 , srv1916456264:84 , srv1971885478:84 , srv2019463567:84 , srv2092946680:84 , srv2097272389:84 , srv243121308:84 , srv260947998:84 , srv278451447:84 , srv331369194:84 , srv369761553:84 , srv390552721:84 , srv399416789:84 , srv461024961:84 , srv461285944:84 , srv466241013:84 , srv488099850:84 , srv524717150:84 , srv56530639:84 , srv576172049:84 , srv625397607:84 , srv630712624:84 , srv700534246:84 , srv722764461:84 , srv810824157:84 , srv903846884:84 , srv986182767:84 , srv1093142466:85 , srv1106360083:85 , srv1110790118:85 , srv1207675652:85 , srv1250452677:85 , srv1281216523:85 , srv1312738254:85 , srv1341927347:85 , srv1371441593:85 , srv1373208727:85 , srv1409934303:85 , srv1434278421:85 , srv1560509146:85 , srv1631893099:85 , srv1728630084:85 , srv1789276299:85 , srv1835564010:85 , srv184776835:85 , srv1871862755:85 , srv1893353846:85 , srv197445714:85 , srv2120619580:85 , srv237676514:85 , srv286671643:85 , srv341350401:85 , srv399277216:85 , srv494677777:85 , srv519176840:85 , srv581144574:85 , srv635741650:85 , srv660757468:85 , srv670799839:85 , srv711880969:85 , srv908433196:85 , srv937966858:85 , srv985672733:85 , srv632427625:86 , srv1135233488:653 }
2023-12-22 15:53:22,165 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=327 (was 277) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 169), AvailableMemoryMB=3661 (was 4255)
2023-12-22 15:53:22,173 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=327, ProcessCount=169, AvailableMemoryMB=3660
2023-12-22 15:53:22,174 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-12-22 15:53:22,175 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-12-22 15:53:22,188 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1030970128:84 , srv1040497437:84 , srv1109078019:84 , srv1113274523:84 , srv1175054435:84 , srv1461766157:84 , srv1475778856:84 , srv1529120338:84 , srv1555529573:84 , srv1695502568:84 , srv1765597481:84 , srv1837700154:84 , srv1839670455:84 , srv1848821760:84 , srv1886950549:84 , srv1921899162:84 , srv1963602499:84 , srv2043872961:84 , srv2140021947:84 , srv225084824:84 , srv231483845:84 , srv245529893:84 , srv39611988:84 , srv463372590:84 , srv568002565:84 , srv892167827:84 , srv90027343:84 , srv918125087:84 , srv972295647:84 , srv9684808:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-12-22 15:53:22,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-22 15:53:22,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1695502568=9, srv1886950549=14, srv1461766157=5, srv892167827=25, srv1109078019=2, srv568002565=24, srv39611988=22, srv972295647=29, srv231483845=20, srv1963602499=16, srv1848821760=13, srv1175054435=4, srv1555529573=8, srv1839670455=12, srv2043872961=17, srv245529893=21, srv1475778856=6, srv2140021947=18, srv1529120338=7, srv90027343=26, srv9684808=28, srv1765597481=10, srv1837700154=11, srv918125087=27, srv463372590=23, srv1113274523=3, srv1921899162=15, srv1030970128=0, srv225084824=19, srv1040497437=1} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-22 15:53:22,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-22 15:53:22,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-22 15:53:22,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-22 15:53:22,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-22 15:53:22,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-22 15:53:22,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-22 15:53:22,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-22 15:53:22,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-22 15:53:22,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-22 15:53:22,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-22 15:53:22,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-22 15:53:22,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-22 15:53:22,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-22 15:53:22,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-22 15:53:22,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-22 15:53:22,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-22 15:53:22,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-22 15:53:22,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-22 15:53:22,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-22 15:53:22,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-22 15:53:22,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-22 15:53:22,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-22 15:53:22,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-22 15:53:22,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-22 15:53:22,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-22 15:53:22,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-22 15:53:22,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-22 15:53:22,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-22 15:53:22,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-22 15:53:22,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-22 15:53:22,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-12-22 15:53:22,227 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1559197213326278, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-22 15:53:22,228 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912684341214951, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1559197213326278, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-12-22 15:55:22,211 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6070498 different iterations.  Found a solution that moves 1845 regions; Going from a computed imbalance of 0.9912684341214951 to a new imbalance of 4.306322040866546E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6833333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-22 15:55:22,218 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1030970128:90 , srv1040497437:90 , srv1109078019:90 , srv1113274523:90 , srv1175054435:90 , srv1461766157:90 , srv1475778856:90 , srv1529120338:90 , srv1555529573:90 , srv1695502568:90 , srv1765597481:90 , srv1837700154:90 , srv1839670455:90 , srv1848821760:90 , srv1886950549:90 , srv1921899162:90 , srv1963602499:90 , srv2043872961:90 , srv2140021947:90 , srv225084824:90 , srv231483845:90 , srv245529893:90 , srv39611988:90 , srv463372590:90 , srv568002565:90 , srv892167827:90 , srv90027343:90 , srv918125087:90 , srv9684808:90 , srv972295647:90 }
2023-12-22 15:55:22,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-22 15:55:22,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1695502568=9, srv1886950549=14, srv1461766157=5, srv892167827=25, srv1109078019=2, srv568002565=24, srv39611988=22, srv972295647=29, srv231483845=20, srv1963602499=16, srv1848821760=13, srv1175054435=4, srv1555529573=8, srv1839670455=12, srv2043872961=17, srv245529893=21, srv1475778856=6, srv2140021947=18, srv1529120338=7, srv90027343=26, srv9684808=28, srv1765597481=10, srv1837700154=11, srv918125087=27, srv463372590=23, srv1113274523=3, srv1921899162=15, srv1030970128=0, srv225084824=19, srv1040497437=1} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-22 15:55:22,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-22 15:55:22,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-22 15:55:22,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-22 15:55:22,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-22 15:55:22,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-22 15:55:22,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-22 15:55:22,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-22 15:55:22,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-22 15:55:22,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-22 15:55:22,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-22 15:55:22,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-22 15:55:22,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-22 15:55:22,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-22 15:55:22,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-22 15:55:22,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-22 15:55:22,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-22 15:55:22,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-22 15:55:22,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-22 15:55:22,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-22 15:55:22,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-22 15:55:22,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-22 15:55:22,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-22 15:55:22,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-22 15:55:22,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-22 15:55:22,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-22 15:55:22,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-22 15:55:22,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-22 15:55:22,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-22 15:55:22,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-22 15:55:22,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-22 15:55:22,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-12-22 15:55:22,252 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-22 15:55:22,253 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1030970128:90 , srv1040497437:90 , srv1109078019:90 , srv1113274523:90 , srv1175054435:90 , srv1461766157:90 , srv1475778856:90 , srv1529120338:90 , srv1555529573:90 , srv1695502568:90 , srv1765597481:90 , srv1837700154:90 , srv1839670455:90 , srv1848821760:90 , srv1886950549:90 , srv1921899162:90 , srv1963602499:90 , srv2043872961:90 , srv2140021947:90 , srv225084824:90 , srv231483845:90 , srv245529893:90 , srv39611988:90 , srv463372590:90 , srv568002565:90 , srv892167827:90 , srv90027343:90 , srv918125087:90 , srv9684808:90 , srv972295647:90 }
2023-12-22 15:55:22,272 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=178 (was 327), ProcessCount=167 (was 169), AvailableMemoryMB=2744 (was 3660)