2024-03-16 11:53:28,481 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2024-03-16 11:53:28,760 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2024-03-16 11:53:28,862 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2024-03-16 11:53:28,922 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-03-16 11:53:28,930 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-03-16 11:53:28,944 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=287, ProcessCount=171, AvailableMemoryMB=3943
2024-03-16 11:53:28,946 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-03-16 11:53:28,946 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-03-16 11:53:29,066 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1001726522:84 , srv1047241842:84 , srv1053188406:84 , srv1059793685:84 , srv1085016487:84 , srv1091047118:84 , srv1093511110:84 , srv1097122796:84 , srv1116338148:84 , srv1125411150:84 , srv1157895288:84 , srv1203914173:84 , srv1209752207:84 , srv1219359357:84 , srv1221097192:84 , srv1249445005:84 , srv127930559:84 , srv1290508324:84 , srv1299893502:84 , srv1304728099:84 , srv1313818915:84 , srv1317349341:84 , srv131996629:84 , srv1346021856:84 , srv1393213587:84 , srv1452780866:84 , srv1457207707:84 , srv1494764037:84 , srv1512602829:84 , srv1521796083:84 , srv1545361361:84 , srv1563078002:84 , srv1563242430:84 , srv1574680133:84 , srv1592469582:84 , srv1594553197:84 , srv1619292252:84 , srv163380578:84 , srv1654967680:84 , srv1672116074:84 , srv1686955848:84 , srv1733250273:84 , srv1736087337:84 , srv1742404659:84 , srv1758175228:84 , srv1758493801:84 , srv1762350161:84 , srv1773207715:84 , srv1786314854:84 , srv1795686433:84 , srv182729939:84 , srv1830142214:84 , srv1844062621:84 , srv1872059275:84 , srv1875388165:84 , srv1890107802:84 , srv1896600788:84 , srv1992982394:84 , srv1998612417:84 , srv2000223357:84 , srv2004075058:84 , srv2041446057:84 , srv2063788881:84 , srv2068806863:84 , srv2074183981:84 , srv2117013246:84 , srv212750098:84 , srv2139280475:84 , srv27542955:84 , srv296893255:84 , srv339433343:84 , srv362987005:84 , srv376088762:84 , srv390629620:84 , srv4111731:84 , srv418414103:84 , srv430893741:84 , srv457767612:84 , srv457851825:84 , srv470755123:84 , srv47323046:84 , srv516360372:84 , srv525321940:84 , srv535090187:84 , srv539445393:84 , srv629649351:84 , srv651456848:84 , srv665286447:84 , srv671768466:84 , srv683618771:84 , srv697744087:84 , srv729436327:84 , srv78299163:84 , srv818167369:84 , srv821009868:84 , srv836022117:84 , srv872680713:84 , srv891566062:84 , srv906892907:84 , srv1406435141:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2024-03-16 11:53:29,073 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-03-16 11:53:29,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1830142214=52, srv339433343=71, srv1773207715=48, srv1512602829=29, srv27542955=69, srv127930559=16, srv1304728099=19, srv2068806863=64, srv131996629=22, srv535090187=84, srv362987005=72, srv1299893502=18, srv418414103=76, srv1594553197=36, srv1219359357=13, srv430893741=77, srv683618771=90, srv1742404659=44, srv2063788881=63, srv1406435141=25, srv1157895288=10, srv665286447=88, srv470755123=80, srv1313818915=20, srv1059793685=3, srv1896600788=57, srv1346021856=23, srv1545361361=31, srv1574680133=34, srv1093511110=6, srv1992982394=58, srv836022117=96, srv1762350161=47, srv629649351=86, srv1758175228=45, srv1317349341=21, srv1494764037=28, srv1736087337=43, srv296893255=70, srv1890107802=56, srv2074183981=65, srv821009868=95, srv457851825=79, srv1001726522=0, srv1116338148=8, srv1998612417=59, srv891566062=98, srv1047241842=1, srv2041446057=62, srv1844062621=53, srv1563242430=33, srv525321940=83, srv163380578=38, srv1290508324=17, srv1672116074=40, srv1053188406=2, srv1452780866=26, srv1786314854=49, srv376088762=73, srv1872059275=54, srv1795686433=50, srv1393213587=24, srv1125411150=9, srv1686955848=41, srv2117013246=66, srv2000223357=60, srv1091047118=5, srv1221097192=14, srv212750098=67, srv1875388165=55, srv1085016487=4, srv47323046=81, srv457767612=78, srv390629620=74, srv1203914173=11, srv697744087=91, srv539445393=85, srv872680713=97, srv1592469582=35, srv1521796083=30, srv818167369=94, srv671768466=89, srv1563078002=32, srv651456848=87, srv1457207707=27, srv1209752207=12, srv1619292252=37, srv1097122796=7, srv4111731=75, srv2004075058=61, srv516360372=82, srv1733250273=42, srv182729939=51, srv2139280475=68, srv1654967680=39, srv906892907=99, srv729436327=92, srv1249445005=15, srv1758493801=46, srv78299163=93} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-03-16 11:53:29,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-03-16 11:53:29,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-03-16 11:53:29,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-03-16 11:53:29,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-03-16 11:53:29,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-03-16 11:53:29,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-03-16 11:53:29,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-03-16 11:53:29,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-03-16 11:53:29,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-03-16 11:53:29,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-03-16 11:53:29,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-03-16 11:53:29,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-03-16 11:53:29,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-03-16 11:53:29,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-03-16 11:53:29,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-03-16 11:53:29,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-03-16 11:53:29,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-03-16 11:53:29,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-03-16 11:53:29,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-03-16 11:53:29,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-03-16 11:53:29,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-03-16 11:53:29,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-03-16 11:53:29,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-03-16 11:53:29,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-03-16 11:53:29,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-03-16 11:53:29,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-03-16 11:53:29,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-03-16 11:53:29,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-03-16 11:53:29,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-03-16 11:53:29,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-03-16 11:53:29,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-03-16 11:53:29,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-03-16 11:53:29,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-03-16 11:53:29,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-03-16 11:53:29,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-03-16 11:53:29,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-03-16 11:53:29,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-03-16 11:53:29,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-03-16 11:53:29,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-03-16 11:53:29,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-03-16 11:53:29,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-03-16 11:53:29,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-03-16 11:53:29,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-03-16 11:53:29,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-03-16 11:53:29,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-03-16 11:53:29,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-03-16 11:53:29,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-03-16 11:53:29,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-03-16 11:53:29,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-03-16 11:53:29,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-03-16 11:53:29,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-03-16 11:53:29,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-03-16 11:53:29,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-03-16 11:53:29,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-03-16 11:53:29,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-03-16 11:53:29,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-03-16 11:53:29,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-03-16 11:53:29,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-03-16 11:53:29,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-03-16 11:53:29,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-03-16 11:53:29,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-03-16 11:53:29,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-03-16 11:53:29,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-03-16 11:53:29,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-03-16 11:53:29,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-03-16 11:53:29,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-03-16 11:53:29,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-03-16 11:53:29,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-03-16 11:53:29,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-03-16 11:53:29,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-03-16 11:53:29,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-03-16 11:53:29,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-03-16 11:53:29,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-03-16 11:53:29,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-03-16 11:53:29,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-03-16 11:53:29,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-03-16 11:53:29,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-03-16 11:53:29,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-03-16 11:53:29,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-03-16 11:53:29,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-03-16 11:53:29,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-03-16 11:53:29,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-03-16 11:53:29,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-03-16 11:53:29,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-03-16 11:53:29,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-03-16 11:53:29,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-03-16 11:53:29,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-03-16 11:53:29,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-03-16 11:53:29,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-03-16 11:53:29,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-03-16 11:53:29,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-03-16 11:53:29,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-03-16 11:53:29,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-03-16 11:53:29,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-03-16 11:53:29,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-03-16 11:53:29,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-03-16 11:53:29,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-03-16 11:53:29,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-03-16 11:53:29,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-03-16 11:53:29,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-03-16 11:53:29,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-03-16 11:53:29,358 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-16 11:53:29,358 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-03-16 11:53:29,358 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-03-16 11:53:29,770 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 582 ms to try 5000 different iterations.  Found a solution that moves 4654 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1880089371553508. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040324472719890266, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0565506834439673, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5171111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040324472719890266, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.18325, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2505, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-16 11:53:29,782 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1896600788:78 , srv339433343:81 , srv1619292252:82 , srv1890107802:82 , srv818167369:82 , srv1053188406:83 , srv1219359357:83 , srv1221097192:83 , srv127930559:83 , srv1592469582:83 , srv1742404659:83 , srv1773207715:83 , srv1992982394:83 , srv1998612417:83 , srv539445393:83 , srv697744087:83 , srv1059793685:84 , srv1116338148:84 , srv1249445005:84 , srv1313818915:84 , srv1574680133:84 , srv1594553197:84 , srv1758493801:84 , srv1762350161:84 , srv1872059275:84 , srv418414103:84 , srv457851825:84 , srv1085016487:85 , srv1203914173:85 , srv1290508324:85 , srv1299893502:85 , srv1521796083:85 , srv163380578:85 , srv1736087337:85 , srv1844062621:85 , srv2074183981:85 , srv376088762:85 , srv457767612:85 , srv470755123:85 , srv516360372:85 , srv535090187:85 , srv651456848:85 , srv836022117:85 , srv1093511110:86 , srv1157895288:86 , srv1209752207:86 , srv1317349341:86 , srv1393213587:86 , srv1457207707:86 , srv1733250273:86 , srv2000223357:86 , srv2041446057:86 , srv296893255:86 , srv665286447:86 , srv671768466:86 , srv683618771:86 , srv872680713:86 , srv1304728099:87 , srv131996629:87 , srv1545361361:87 , srv1786314854:87 , srv1795686433:87 , srv182729939:87 , srv1830142214:87 , srv2068806863:87 , srv212750098:87 , srv2139280475:87 , srv729436327:87 , srv78299163:87 , srv821009868:87 , srv891566062:87 , srv1452780866:88 , srv1563078002:88 , srv1654967680:88 , srv2004075058:88 , srv27542955:88 , srv1047241842:89 , srv1125411150:89 , srv1672116074:89 , srv1686955848:89 , srv1758175228:89 , srv362987005:89 , srv525321940:89 , srv1001726522:90 , srv1346021856:90 , srv1494764037:90 , srv2117013246:90 , srv390629620:90 , srv47323046:90 , srv629649351:90 , srv1097122796:91 , srv1512602829:91 , srv1875388165:91 , srv1091047118:92 , srv1563242430:92 , srv2063788881:93 , srv430893741:93 , srv4111731:96 , srv906892907:96 , srv1406435141:448 }
2024-03-16 11:53:29,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-03-16 11:53:29,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1830142214=52, srv339433343=71, srv1773207715=48, srv1512602829=29, srv27542955=69, srv127930559=16, srv1304728099=19, srv2068806863=64, srv131996629=22, srv535090187=84, srv362987005=72, srv1299893502=18, srv418414103=76, srv1594553197=36, srv1219359357=13, srv430893741=77, srv683618771=90, srv1742404659=44, srv2063788881=63, srv1406435141=25, srv1157895288=10, srv665286447=88, srv470755123=80, srv1313818915=20, srv1059793685=3, srv1896600788=57, srv1346021856=23, srv1545361361=31, srv1574680133=34, srv1093511110=6, srv1992982394=58, srv836022117=96, srv1762350161=47, srv629649351=86, srv1758175228=45, srv1317349341=21, srv1494764037=28, srv1736087337=43, srv296893255=70, srv1890107802=56, srv2074183981=65, srv821009868=95, srv457851825=79, srv1001726522=0, srv1116338148=8, srv1998612417=59, srv891566062=98, srv1047241842=1, srv2041446057=62, srv1844062621=53, srv1563242430=33, srv525321940=83, srv163380578=38, srv1290508324=17, srv1672116074=40, srv1053188406=2, srv1452780866=26, srv1786314854=49, srv376088762=73, srv1872059275=54, srv1795686433=50, srv1393213587=24, srv1125411150=9, srv1686955848=41, srv2117013246=66, srv2000223357=60, srv1091047118=5, srv1221097192=14, srv212750098=67, srv1875388165=55, srv1085016487=4, srv47323046=81, srv457767612=78, srv390629620=74, srv1203914173=11, srv697744087=91, srv539445393=85, srv872680713=97, srv1592469582=35, srv1521796083=30, srv818167369=94, srv671768466=89, srv1563078002=32, srv651456848=87, srv1457207707=27, srv1209752207=12, srv1619292252=37, srv1097122796=7, srv4111731=75, srv2004075058=61, srv516360372=82, srv1733250273=42, srv182729939=51, srv2139280475=68, srv1654967680=39, srv906892907=99, srv729436327=92, srv1249445005=15, srv1758493801=46, srv78299163=93} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-03-16 11:53:29,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-03-16 11:53:29,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-03-16 11:53:29,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-03-16 11:53:29,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-03-16 11:53:29,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-03-16 11:53:29,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-03-16 11:53:29,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-03-16 11:53:29,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-03-16 11:53:29,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-03-16 11:53:29,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-03-16 11:53:29,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-03-16 11:53:29,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-03-16 11:53:29,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-03-16 11:53:29,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-03-16 11:53:29,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-03-16 11:53:29,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-03-16 11:53:29,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-03-16 11:53:29,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-03-16 11:53:29,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-03-16 11:53:29,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-03-16 11:53:29,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-03-16 11:53:29,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-03-16 11:53:29,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-03-16 11:53:29,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-03-16 11:53:29,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-03-16 11:53:29,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-03-16 11:53:29,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-03-16 11:53:29,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-03-16 11:53:29,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-03-16 11:53:29,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-03-16 11:53:29,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-03-16 11:53:29,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-03-16 11:53:29,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-03-16 11:53:29,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-03-16 11:53:29,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-03-16 11:53:29,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-03-16 11:53:29,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-03-16 11:53:29,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-03-16 11:53:29,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-03-16 11:53:29,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-03-16 11:53:29,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-03-16 11:53:29,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-03-16 11:53:29,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-03-16 11:53:29,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-03-16 11:53:29,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-03-16 11:53:29,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-03-16 11:53:29,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-03-16 11:53:29,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-03-16 11:53:29,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-03-16 11:53:29,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-03-16 11:53:29,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-03-16 11:53:29,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-03-16 11:53:29,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-03-16 11:53:29,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-03-16 11:53:29,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-03-16 11:53:29,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-03-16 11:53:29,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-03-16 11:53:29,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-03-16 11:53:29,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-03-16 11:53:29,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-03-16 11:53:29,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-03-16 11:53:29,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-03-16 11:53:29,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-03-16 11:53:29,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-03-16 11:53:29,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-03-16 11:53:29,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-03-16 11:53:29,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-03-16 11:53:29,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-03-16 11:53:29,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-03-16 11:53:29,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-03-16 11:53:29,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-03-16 11:53:29,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-03-16 11:53:29,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-03-16 11:53:29,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-03-16 11:53:29,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-03-16 11:53:29,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-03-16 11:53:29,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-03-16 11:53:29,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-03-16 11:53:29,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-03-16 11:53:29,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-03-16 11:53:29,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-03-16 11:53:29,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-03-16 11:53:29,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-03-16 11:53:29,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-03-16 11:53:29,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-03-16 11:53:29,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-03-16 11:53:29,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-03-16 11:53:29,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-03-16 11:53:29,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-03-16 11:53:29,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-03-16 11:53:29,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-03-16 11:53:29,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-03-16 11:53:29,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-03-16 11:53:29,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-03-16 11:53:29,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-03-16 11:53:29,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-03-16 11:53:29,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-03-16 11:53:29,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-03-16 11:53:29,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-03-16 11:53:29,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-03-16 11:53:29,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-03-16 11:53:29,948 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040324472719890266, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0565506834439673, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040324472719890266, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.18325, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2505, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-16 11:53:29,948 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-03-16 11:53:29,948 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18797634915083342, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040324472719890266, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0565506834439673, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040324472719890266, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.18325, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2505, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-03-16 11:53:30,231 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 403 ms to try 5000 different iterations.  Found a solution that moves 2677 regions; Going from a computed imbalance of 0.18797634915083342 to a new imbalance of 0.016276286741505573. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022060014052070618); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03640710084782443, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29744444444444446, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022060014052070618); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012666666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.050916666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-16 11:53:30,240 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv4111731:76 , srv2063788881:77 , srv430893741:77 , srv906892907:78 , srv1097122796:79 , srv1091047118:80 , srv1563242430:80 , srv1875388165:80 , srv390629620:81 , srv1001726522:82 , srv1512602829:82 , srv1545361361:82 , srv1654967680:82 , srv1672116074:82 , srv1686955848:82 , srv2004075058:82 , srv47323046:82 , srv525321940:82 , srv1047241842:83 , srv1452780866:83 , srv1494764037:83 , srv1758175228:83 , srv1786314854:83 , srv1795686433:83 , srv2068806863:83 , srv629649351:83 , srv78299163:83 , srv1125411150:84 , srv1304728099:84 , srv1346021856:84 , srv182729939:84 , srv1830142214:84 , srv27542955:84 , srv362987005:84 , srv1203914173:85 , srv1317349341:85 , srv131996629:85 , srv1393213587:85 , srv1844062621:85 , srv2041446057:85 , srv2117013246:85 , srv212750098:85 , srv2139280475:85 , srv457767612:85 , srv1249445005:86 , srv1563078002:86 , srv163380578:86 , srv1992982394:86 , srv2074183981:86 , srv470755123:86 , srv535090187:86 , srv651456848:86 , srv683618771:86 , srv729436327:86 , srv821009868:86 , srv836022117:86 , srv1093511110:87 , srv1209752207:87 , srv1457207707:87 , srv1521796083:87 , srv1762350161:87 , srv296893255:87 , srv671768466:87 , srv872680713:87 , srv891566062:87 , srv1053188406:88 , srv1116338148:88 , srv1219359357:88 , srv127930559:88 , srv1290508324:88 , srv1299893502:88 , srv1594553197:88 , srv1733250273:88 , srv1758493801:88 , srv1773207715:88 , srv457851825:88 , srv539445393:88 , srv665286447:88 , srv697744087:88 , srv1059793685:89 , srv1085016487:89 , srv1221097192:89 , srv1313818915:89 , srv1574680133:89 , srv1592469582:89 , srv1736087337:89 , srv1998612417:89 , srv2000223357:89 , srv376088762:89 , srv516360372:89 , srv1157895288:90 , srv1742404659:90 , srv418414103:90 , srv1619292252:91 , srv339433343:91 , srv818167369:91 , srv1872059275:92 , srv1890107802:93 , srv1896600788:93 , srv1406435141:522 }
2024-03-16 11:53:30,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-03-16 11:53:30,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1830142214=52, srv339433343=71, srv1773207715=48, srv1512602829=29, srv27542955=69, srv127930559=16, srv1304728099=19, srv2068806863=64, srv131996629=22, srv535090187=84, srv362987005=72, srv1299893502=18, srv418414103=76, srv1594553197=36, srv1219359357=13, srv430893741=77, srv683618771=90, srv1742404659=44, srv2063788881=63, srv1406435141=25, srv1157895288=10, srv665286447=88, srv470755123=80, srv1313818915=20, srv1059793685=3, srv1896600788=57, srv1346021856=23, srv1545361361=31, srv1574680133=34, srv1093511110=6, srv1992982394=58, srv836022117=96, srv1762350161=47, srv629649351=86, srv1758175228=45, srv1317349341=21, srv1494764037=28, srv1736087337=43, srv296893255=70, srv1890107802=56, srv2074183981=65, srv821009868=95, srv457851825=79, srv1001726522=0, srv1116338148=8, srv1998612417=59, srv891566062=98, srv1047241842=1, srv2041446057=62, srv1844062621=53, srv1563242430=33, srv525321940=83, srv163380578=38, srv1290508324=17, srv1672116074=40, srv1053188406=2, srv1452780866=26, srv1786314854=49, srv376088762=73, srv1872059275=54, srv1795686433=50, srv1393213587=24, srv1125411150=9, srv1686955848=41, srv2117013246=66, srv2000223357=60, srv1091047118=5, srv1221097192=14, srv212750098=67, srv1875388165=55, srv1085016487=4, srv47323046=81, srv457767612=78, srv390629620=74, srv1203914173=11, srv697744087=91, srv539445393=85, srv872680713=97, srv1592469582=35, srv1521796083=30, srv818167369=94, srv671768466=89, srv1563078002=32, srv651456848=87, srv1457207707=27, srv1209752207=12, srv1619292252=37, srv1097122796=7, srv4111731=75, srv2004075058=61, srv516360372=82, srv1733250273=42, srv182729939=51, srv2139280475=68, srv1654967680=39, srv906892907=99, srv729436327=92, srv1249445005=15, srv1758493801=46, srv78299163=93} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-03-16 11:53:30,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-03-16 11:53:30,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-03-16 11:53:30,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-03-16 11:53:30,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-03-16 11:53:30,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-03-16 11:53:30,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-03-16 11:53:30,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-03-16 11:53:30,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-03-16 11:53:30,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-03-16 11:53:30,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-03-16 11:53:30,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-03-16 11:53:30,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-03-16 11:53:30,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-03-16 11:53:30,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-03-16 11:53:30,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-03-16 11:53:30,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-03-16 11:53:30,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-03-16 11:53:30,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-03-16 11:53:30,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-03-16 11:53:30,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-03-16 11:53:30,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-03-16 11:53:30,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-03-16 11:53:30,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-03-16 11:53:30,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-03-16 11:53:30,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-03-16 11:53:30,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-03-16 11:53:30,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-03-16 11:53:30,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-03-16 11:53:30,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-03-16 11:53:30,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-03-16 11:53:30,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-03-16 11:53:30,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-03-16 11:53:30,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-03-16 11:53:30,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-03-16 11:53:30,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-03-16 11:53:30,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-03-16 11:53:30,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-03-16 11:53:30,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-03-16 11:53:30,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-03-16 11:53:30,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-03-16 11:53:30,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-03-16 11:53:30,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-03-16 11:53:30,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-03-16 11:53:30,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-03-16 11:53:30,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-03-16 11:53:30,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-03-16 11:53:30,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-03-16 11:53:30,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-03-16 11:53:30,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-03-16 11:53:30,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-03-16 11:53:30,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-03-16 11:53:30,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-03-16 11:53:30,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-03-16 11:53:30,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-03-16 11:53:30,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-03-16 11:53:30,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-03-16 11:53:30,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-03-16 11:53:30,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-03-16 11:53:30,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-03-16 11:53:30,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-03-16 11:53:30,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-03-16 11:53:30,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-03-16 11:53:30,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-03-16 11:53:30,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-03-16 11:53:30,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-03-16 11:53:30,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-03-16 11:53:30,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-03-16 11:53:30,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-03-16 11:53:30,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-03-16 11:53:30,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-03-16 11:53:30,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-03-16 11:53:30,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-03-16 11:53:30,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-03-16 11:53:30,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-03-16 11:53:30,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-03-16 11:53:30,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-03-16 11:53:30,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-03-16 11:53:30,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-03-16 11:53:30,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-03-16 11:53:30,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-03-16 11:53:30,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-03-16 11:53:30,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-03-16 11:53:30,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-03-16 11:53:30,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-03-16 11:53:30,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-03-16 11:53:30,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-03-16 11:53:30,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-03-16 11:53:30,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-03-16 11:53:30,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-03-16 11:53:30,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-03-16 11:53:30,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-03-16 11:53:30,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-03-16 11:53:30,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-03-16 11:53:30,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-03-16 11:53:30,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-03-16 11:53:30,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-03-16 11:53:30,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-03-16 11:53:30,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-03-16 11:53:30,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-03-16 11:53:30,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-03-16 11:53:30,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-03-16 11:53:30,397 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022060014052070618); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03640710084782443, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022060014052070618); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012666666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.050916666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-16 11:53:30,397 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-03-16 11:53:30,397 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.0162575419868659, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022060014052070618); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03640710084782443, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022060014052070618); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012666666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.050916666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-03-16 11:53:30,767 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 487 ms to try 5000 different iterations.  Found a solution that moves 1081 regions; Going from a computed imbalance of 0.0162575419868659 to a new imbalance of 8.163372015060989E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00588156798823659); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.011711074693750519); MoveCostFunction : (multiplier=7.0, imbalance=0.12011111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.00588156798823659); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003916666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-16 11:53:30,769 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1157895288:81 , srv1872059275:81 , srv2000223357:81 , srv1085016487:83 , srv1346021856:83 , srv1563078002:83 , srv1890107802:83 , srv2117013246:83 , srv376088762:83 , srv629649351:83 , srv665286447:83 , srv1093511110:84 , srv1313818915:84 , srv131996629:84 , srv1512602829:84 , srv1521796083:84 , srv1619292252:84 , srv1733250273:84 , srv1736087337:84 , srv296893255:84 , srv418414103:84 , srv516360372:84 , srv671768466:84 , srv729436327:84 , srv818167369:84 , srv891566062:84 , srv906892907:84 , srv1059793685:85 , srv1091047118:85 , srv1290508324:85 , srv1299893502:85 , srv1457207707:85 , srv1494764037:85 , srv1574680133:85 , srv1686955848:85 , srv1742404659:85 , srv1758493801:85 , srv1998612417:85 , srv212750098:85 , srv339433343:85 , srv457851825:85 , srv821009868:85 , srv872680713:85 , srv1001726522:86 , srv1047241842:86 , srv1053188406:86 , srv1116338148:86 , srv1125411150:86 , srv1209752207:86 , srv1219359357:86 , srv1221097192:86 , srv1317349341:86 , srv1393213587:86 , srv1563242430:86 , srv1592469582:86 , srv1594553197:86 , srv1672116074:86 , srv1758175228:86 , srv1773207715:86 , srv182729939:86 , srv1875388165:86 , srv1896600788:86 , srv2041446057:86 , srv2139280475:86 , srv27542955:86 , srv362987005:86 , srv4111731:86 , srv525321940:86 , srv535090187:86 , srv539445393:86 , srv651456848:86 , srv683618771:86 , srv836022117:86 , srv1097122796:87 , srv1203914173:87 , srv1249445005:87 , srv127930559:87 , srv1304728099:87 , srv1452780866:87 , srv163380578:87 , srv1654967680:87 , srv1762350161:87 , srv1786314854:87 , srv1830142214:87 , srv2004075058:87 , srv2063788881:87 , srv2068806863:87 , srv2074183981:87 , srv390629620:87 , srv430893741:87 , srv470755123:87 , srv47323046:87 , srv697744087:87 , srv1545361361:88 , srv1795686433:88 , srv1844062621:88 , srv457767612:88 , srv78299163:88 , srv1992982394:89 , srv1406435141:540 }
2024-03-16 11:53:30,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-03-16 11:53:30,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1830142214=52, srv339433343=71, srv1773207715=48, srv1512602829=29, srv27542955=69, srv127930559=16, srv1304728099=19, srv2068806863=64, srv131996629=22, srv535090187=84, srv362987005=72, srv1299893502=18, srv418414103=76, srv1594553197=36, srv1219359357=13, srv430893741=77, srv683618771=90, srv1742404659=44, srv2063788881=63, srv1406435141=25, srv1157895288=10, srv665286447=88, srv470755123=80, srv1313818915=20, srv1059793685=3, srv1896600788=57, srv1346021856=23, srv1545361361=31, srv1574680133=34, srv1093511110=6, srv1992982394=58, srv836022117=96, srv1762350161=47, srv629649351=86, srv1758175228=45, srv1317349341=21, srv1494764037=28, srv1736087337=43, srv296893255=70, srv1890107802=56, srv2074183981=65, srv821009868=95, srv457851825=79, srv1001726522=0, srv1116338148=8, srv1998612417=59, srv891566062=98, srv1047241842=1, srv2041446057=62, srv1844062621=53, srv1563242430=33, srv525321940=83, srv163380578=38, srv1290508324=17, srv1672116074=40, srv1053188406=2, srv1452780866=26, srv1786314854=49, srv376088762=73, srv1872059275=54, srv1795686433=50, srv1393213587=24, srv1125411150=9, srv1686955848=41, srv2117013246=66, srv2000223357=60, srv1091047118=5, srv1221097192=14, srv212750098=67, srv1875388165=55, srv1085016487=4, srv47323046=81, srv457767612=78, srv390629620=74, srv1203914173=11, srv697744087=91, srv539445393=85, srv872680713=97, srv1592469582=35, srv1521796083=30, srv818167369=94, srv671768466=89, srv1563078002=32, srv651456848=87, srv1457207707=27, srv1209752207=12, srv1619292252=37, srv1097122796=7, srv4111731=75, srv2004075058=61, srv516360372=82, srv1733250273=42, srv182729939=51, srv2139280475=68, srv1654967680=39, srv906892907=99, srv729436327=92, srv1249445005=15, srv1758493801=46, srv78299163=93} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-03-16 11:53:30,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-03-16 11:53:30,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-03-16 11:53:30,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-03-16 11:53:30,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-03-16 11:53:30,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-03-16 11:53:30,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-03-16 11:53:30,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-03-16 11:53:30,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-03-16 11:53:30,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-03-16 11:53:30,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-03-16 11:53:30,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-03-16 11:53:30,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-03-16 11:53:30,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-03-16 11:53:30,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-03-16 11:53:30,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-03-16 11:53:30,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-03-16 11:53:30,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-03-16 11:53:30,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-03-16 11:53:30,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-03-16 11:53:30,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-03-16 11:53:30,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-03-16 11:53:30,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-03-16 11:53:30,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-03-16 11:53:30,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-03-16 11:53:30,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-03-16 11:53:30,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-03-16 11:53:30,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-03-16 11:53:30,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-03-16 11:53:30,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-03-16 11:53:30,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-03-16 11:53:30,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-03-16 11:53:30,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-03-16 11:53:30,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-03-16 11:53:30,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-03-16 11:53:30,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-03-16 11:53:30,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-03-16 11:53:30,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-03-16 11:53:30,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-03-16 11:53:30,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-03-16 11:53:30,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-03-16 11:53:30,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-03-16 11:53:30,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-03-16 11:53:30,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-03-16 11:53:30,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-03-16 11:53:30,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-03-16 11:53:30,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-03-16 11:53:30,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-03-16 11:53:30,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-03-16 11:53:30,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-03-16 11:53:30,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-03-16 11:53:30,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-03-16 11:53:30,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-03-16 11:53:30,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-03-16 11:53:30,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-03-16 11:53:30,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-03-16 11:53:30,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-03-16 11:53:30,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-03-16 11:53:30,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-03-16 11:53:30,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-03-16 11:53:30,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-03-16 11:53:30,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-03-16 11:53:30,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-03-16 11:53:30,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-03-16 11:53:30,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-03-16 11:53:30,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-03-16 11:53:30,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-03-16 11:53:30,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-03-16 11:53:30,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-03-16 11:53:30,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-03-16 11:53:30,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-03-16 11:53:30,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-03-16 11:53:30,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-03-16 11:53:30,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-03-16 11:53:30,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-03-16 11:53:30,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-03-16 11:53:30,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-03-16 11:53:30,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-03-16 11:53:30,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-03-16 11:53:30,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-03-16 11:53:30,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-03-16 11:53:30,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-03-16 11:53:30,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-03-16 11:53:30,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-03-16 11:53:30,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-03-16 11:53:30,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-03-16 11:53:30,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-03-16 11:53:30,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-03-16 11:53:30,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-03-16 11:53:30,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-03-16 11:53:30,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-03-16 11:53:30,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-03-16 11:53:30,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-03-16 11:53:30,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-03-16 11:53:30,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-03-16 11:53:30,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-03-16 11:53:30,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-03-16 11:53:30,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-03-16 11:53:30,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-03-16 11:53:30,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-03-16 11:53:30,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-03-16 11:53:30,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-03-16 11:53:30,937 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00588156798823659); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.011711074693750519); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.00588156798823659); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003916666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-16 11:53:30,937 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-03-16 11:53:30,937 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=8.087678777236976E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00588156798823659); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.011711074693750519); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.00588156798823659); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003916666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-03-16 11:53:31,185 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 385 ms to try 5000 different iterations.  Found a solution that moves 276 regions; Going from a computed imbalance of 8.087678777236976E-4 to a new imbalance of 2.8487242809515143E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.178341294946416E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002118805753879094); MoveCostFunction : (multiplier=7.0, imbalance=0.030666666666666665, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.178341294946416E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-16 11:53:31,186 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1047241842:83 , srv1125411150:83 , srv1209752207:83 , srv1290508324:83 , srv47323046:83 , srv872680713:83 , srv1001726522:84 , srv1059793685:84 , srv1116338148:84 , srv1219359357:84 , srv1221097192:84 , srv127930559:84 , srv1299893502:84 , srv1304728099:84 , srv1317349341:84 , srv1452780866:84 , srv1457207707:84 , srv1494764037:84 , srv1563242430:84 , srv1574680133:84 , srv1594553197:84 , srv163380578:84 , srv1733250273:84 , srv1736087337:84 , srv1742404659:84 , srv1758175228:84 , srv1762350161:84 , srv1795686433:84 , srv1830142214:84 , srv1844062621:84 , srv1875388165:84 , srv1890107802:84 , srv1992982394:84 , srv2074183981:84 , srv2117013246:84 , srv2139280475:84 , srv27542955:84 , srv339433343:84 , srv362987005:84 , srv390629620:84 , srv4111731:84 , srv418414103:84 , srv457767612:84 , srv470755123:84 , srv516360372:84 , srv697744087:84 , srv78299163:84 , srv821009868:84 , srv906892907:84 , srv1053188406:85 , srv1085016487:85 , srv1091047118:85 , srv1093511110:85 , srv1097122796:85 , srv1203914173:85 , srv1249445005:85 , srv1313818915:85 , srv1346021856:85 , srv1393213587:85 , srv1512602829:85 , srv1545361361:85 , srv1563078002:85 , srv1592469582:85 , srv1619292252:85 , srv1654967680:85 , srv1758493801:85 , srv1773207715:85 , srv1786314854:85 , srv182729939:85 , srv1872059275:85 , srv1896600788:85 , srv1998612417:85 , srv2004075058:85 , srv2041446057:85 , srv2063788881:85 , srv2068806863:85 , srv212750098:85 , srv296893255:85 , srv376088762:85 , srv430893741:85 , srv457851825:85 , srv525321940:85 , srv535090187:85 , srv539445393:85 , srv651456848:85 , srv671768466:85 , srv683618771:85 , srv729436327:85 , srv818167369:85 , srv836022117:85 , srv891566062:85 , srv1157895288:86 , srv131996629:86 , srv1521796083:86 , srv1672116074:86 , srv1686955848:86 , srv2000223357:86 , srv629649351:86 , srv665286447:86 , srv1406435141:632 }
2024-03-16 11:53:31,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-03-16 11:53:31,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1830142214=52, srv339433343=71, srv1773207715=48, srv1512602829=29, srv27542955=69, srv127930559=16, srv1304728099=19, srv2068806863=64, srv131996629=22, srv535090187=84, srv362987005=72, srv1299893502=18, srv418414103=76, srv1594553197=36, srv1219359357=13, srv430893741=77, srv683618771=90, srv1742404659=44, srv2063788881=63, srv1406435141=25, srv1157895288=10, srv665286447=88, srv470755123=80, srv1313818915=20, srv1059793685=3, srv1896600788=57, srv1346021856=23, srv1545361361=31, srv1574680133=34, srv1093511110=6, srv1992982394=58, srv836022117=96, srv1762350161=47, srv629649351=86, srv1758175228=45, srv1317349341=21, srv1494764037=28, srv1736087337=43, srv296893255=70, srv1890107802=56, srv2074183981=65, srv821009868=95, srv457851825=79, srv1001726522=0, srv1116338148=8, srv1998612417=59, srv891566062=98, srv1047241842=1, srv2041446057=62, srv1844062621=53, srv1563242430=33, srv525321940=83, srv163380578=38, srv1290508324=17, srv1672116074=40, srv1053188406=2, srv1452780866=26, srv1786314854=49, srv376088762=73, srv1872059275=54, srv1795686433=50, srv1393213587=24, srv1125411150=9, srv1686955848=41, srv2117013246=66, srv2000223357=60, srv1091047118=5, srv1221097192=14, srv212750098=67, srv1875388165=55, srv1085016487=4, srv47323046=81, srv457767612=78, srv390629620=74, srv1203914173=11, srv697744087=91, srv539445393=85, srv872680713=97, srv1592469582=35, srv1521796083=30, srv818167369=94, srv671768466=89, srv1563078002=32, srv651456848=87, srv1457207707=27, srv1209752207=12, srv1619292252=37, srv1097122796=7, srv4111731=75, srv2004075058=61, srv516360372=82, srv1733250273=42, srv182729939=51, srv2139280475=68, srv1654967680=39, srv906892907=99, srv729436327=92, srv1249445005=15, srv1758493801=46, srv78299163=93} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-03-16 11:53:31,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-03-16 11:53:31,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-03-16 11:53:31,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-03-16 11:53:31,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-03-16 11:53:31,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-03-16 11:53:31,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-03-16 11:53:31,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-03-16 11:53:31,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-03-16 11:53:31,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-03-16 11:53:31,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-03-16 11:53:31,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-03-16 11:53:31,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-03-16 11:53:31,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-03-16 11:53:31,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-03-16 11:53:31,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-03-16 11:53:31,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-03-16 11:53:31,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-03-16 11:53:31,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-03-16 11:53:31,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-03-16 11:53:31,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-03-16 11:53:31,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-03-16 11:53:31,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-03-16 11:53:31,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-03-16 11:53:31,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-03-16 11:53:31,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-03-16 11:53:31,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-03-16 11:53:31,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-03-16 11:53:31,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-03-16 11:53:31,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-03-16 11:53:31,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-03-16 11:53:31,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-03-16 11:53:31,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-03-16 11:53:31,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-03-16 11:53:31,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-03-16 11:53:31,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-03-16 11:53:31,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-03-16 11:53:31,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-03-16 11:53:31,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-03-16 11:53:31,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-03-16 11:53:31,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-03-16 11:53:31,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-03-16 11:53:31,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-03-16 11:53:31,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-03-16 11:53:31,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-03-16 11:53:31,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-03-16 11:53:31,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-03-16 11:53:31,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-03-16 11:53:31,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-03-16 11:53:31,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-03-16 11:53:31,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-03-16 11:53:31,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-03-16 11:53:31,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-03-16 11:53:31,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-03-16 11:53:31,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-03-16 11:53:31,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-03-16 11:53:31,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-03-16 11:53:31,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-03-16 11:53:31,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-03-16 11:53:31,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-03-16 11:53:31,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-03-16 11:53:31,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-03-16 11:53:31,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-03-16 11:53:31,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-03-16 11:53:31,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-03-16 11:53:31,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-03-16 11:53:31,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-03-16 11:53:31,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-03-16 11:53:31,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-03-16 11:53:31,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-03-16 11:53:31,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-03-16 11:53:31,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-03-16 11:53:31,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-03-16 11:53:31,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-03-16 11:53:31,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-03-16 11:53:31,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-03-16 11:53:31,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-03-16 11:53:31,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-03-16 11:53:31,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-03-16 11:53:31,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-03-16 11:53:31,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-03-16 11:53:31,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-03-16 11:53:31,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-03-16 11:53:31,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-03-16 11:53:31,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-03-16 11:53:31,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-03-16 11:53:31,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-03-16 11:53:31,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-03-16 11:53:31,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-03-16 11:53:31,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-03-16 11:53:31,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-03-16 11:53:31,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-03-16 11:53:31,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-03-16 11:53:31,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-03-16 11:53:31,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-03-16 11:53:31,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-03-16 11:53:31,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-03-16 11:53:31,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-03-16 11:53:31,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-03-16 11:53:31,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-03-16 11:53:31,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-03-16 11:53:31,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-03-16 11:53:31,318 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.178341294946416E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002118805753879094); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.178341294946416E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-16 11:53:31,318 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-03-16 11:53:31,318 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=2.6554649503370158E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.178341294946416E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002118805753879094); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.178341294946416E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-03-16 11:53:31,556 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 340 ms to try 5000 different iterations.  Found a solution that moves 21 regions; Going from a computed imbalance of 2.6554649503370158E-5 to a new imbalance of 8.40680912885026E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.0023333333333333335); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-16 11:53:31,556 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1157895288:83 , srv1592469582:83 , srv1672116074:83 , srv362987005:83 , srv665286447:83 , srv683618771:83 , srv891566062:83 , srv1001726522:84 , srv1053188406:84 , srv1059793685:84 , srv1085016487:84 , srv1091047118:84 , srv1093511110:84 , srv1097122796:84 , srv1116338148:84 , srv1125411150:84 , srv1203914173:84 , srv1209752207:84 , srv1221097192:84 , srv1249445005:84 , srv127930559:84 , srv1299893502:84 , srv1304728099:84 , srv1313818915:84 , srv131996629:84 , srv1346021856:84 , srv1393213587:84 , srv1452780866:84 , srv1457207707:84 , srv1494764037:84 , srv1512602829:84 , srv1521796083:84 , srv1545361361:84 , srv1563078002:84 , srv1563242430:84 , srv1574680133:84 , srv1594553197:84 , srv1619292252:84 , srv163380578:84 , srv1654967680:84 , srv1686955848:84 , srv1733250273:84 , srv1736087337:84 , srv1742404659:84 , srv1758175228:84 , srv1758493801:84 , srv1762350161:84 , srv1773207715:84 , srv1786314854:84 , srv1795686433:84 , srv182729939:84 , srv1830142214:84 , srv1844062621:84 , srv1872059275:84 , srv1890107802:84 , srv1896600788:84 , srv1992982394:84 , srv1998612417:84 , srv2000223357:84 , srv2004075058:84 , srv2041446057:84 , srv2063788881:84 , srv2068806863:84 , srv2074183981:84 , srv2117013246:84 , srv212750098:84 , srv2139280475:84 , srv27542955:84 , srv296893255:84 , srv376088762:84 , srv390629620:84 , srv4111731:84 , srv418414103:84 , srv430893741:84 , srv457767612:84 , srv457851825:84 , srv470755123:84 , srv47323046:84 , srv516360372:84 , srv525321940:84 , srv535090187:84 , srv539445393:84 , srv629649351:84 , srv651456848:84 , srv671768466:84 , srv697744087:84 , srv729436327:84 , srv78299163:84 , srv818167369:84 , srv821009868:84 , srv836022117:84 , srv906892907:84 , srv1047241842:85 , srv1219359357:85 , srv1290508324:85 , srv1317349341:85 , srv1875388165:85 , srv339433343:85 , srv872680713:85 , srv1406435141:684 }
2024-03-16 11:53:31,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-03-16 11:53:31,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1830142214=52, srv339433343=71, srv1773207715=48, srv1512602829=29, srv27542955=69, srv127930559=16, srv1304728099=19, srv2068806863=64, srv131996629=22, srv535090187=84, srv362987005=72, srv1299893502=18, srv418414103=76, srv1594553197=36, srv1219359357=13, srv430893741=77, srv683618771=90, srv1742404659=44, srv2063788881=63, srv1406435141=25, srv1157895288=10, srv665286447=88, srv470755123=80, srv1313818915=20, srv1059793685=3, srv1896600788=57, srv1346021856=23, srv1545361361=31, srv1574680133=34, srv1093511110=6, srv1992982394=58, srv836022117=96, srv1762350161=47, srv629649351=86, srv1758175228=45, srv1317349341=21, srv1494764037=28, srv1736087337=43, srv296893255=70, srv1890107802=56, srv2074183981=65, srv821009868=95, srv457851825=79, srv1001726522=0, srv1116338148=8, srv1998612417=59, srv891566062=98, srv1047241842=1, srv2041446057=62, srv1844062621=53, srv1563242430=33, srv525321940=83, srv163380578=38, srv1290508324=17, srv1672116074=40, srv1053188406=2, srv1452780866=26, srv1786314854=49, srv376088762=73, srv1872059275=54, srv1795686433=50, srv1393213587=24, srv1125411150=9, srv1686955848=41, srv2117013246=66, srv2000223357=60, srv1091047118=5, srv1221097192=14, srv212750098=67, srv1875388165=55, srv1085016487=4, srv47323046=81, srv457767612=78, srv390629620=74, srv1203914173=11, srv697744087=91, srv539445393=85, srv872680713=97, srv1592469582=35, srv1521796083=30, srv818167369=94, srv671768466=89, srv1563078002=32, srv651456848=87, srv1457207707=27, srv1209752207=12, srv1619292252=37, srv1097122796=7, srv4111731=75, srv2004075058=61, srv516360372=82, srv1733250273=42, srv182729939=51, srv2139280475=68, srv1654967680=39, srv906892907=99, srv729436327=92, srv1249445005=15, srv1758493801=46, srv78299163=93} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-03-16 11:53:31,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-03-16 11:53:31,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-03-16 11:53:31,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-03-16 11:53:31,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-03-16 11:53:31,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-03-16 11:53:31,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-03-16 11:53:31,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-03-16 11:53:31,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-03-16 11:53:31,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-03-16 11:53:31,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-03-16 11:53:31,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-03-16 11:53:31,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-03-16 11:53:31,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-03-16 11:53:31,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-03-16 11:53:31,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-03-16 11:53:31,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-03-16 11:53:31,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-03-16 11:53:31,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-03-16 11:53:31,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-03-16 11:53:31,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-03-16 11:53:31,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-03-16 11:53:31,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-03-16 11:53:31,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-03-16 11:53:31,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-03-16 11:53:31,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-03-16 11:53:31,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-03-16 11:53:31,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-03-16 11:53:31,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-03-16 11:53:31,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-03-16 11:53:31,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-03-16 11:53:31,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-03-16 11:53:31,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-03-16 11:53:31,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-03-16 11:53:31,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-03-16 11:53:31,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-03-16 11:53:31,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-03-16 11:53:31,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-03-16 11:53:31,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-03-16 11:53:31,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-03-16 11:53:31,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-03-16 11:53:31,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-03-16 11:53:31,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-03-16 11:53:31,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-03-16 11:53:31,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-03-16 11:53:31,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-03-16 11:53:31,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-03-16 11:53:31,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-03-16 11:53:31,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-03-16 11:53:31,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-03-16 11:53:31,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-03-16 11:53:31,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-03-16 11:53:31,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-03-16 11:53:31,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-03-16 11:53:31,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-03-16 11:53:31,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-03-16 11:53:31,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-03-16 11:53:31,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-03-16 11:53:31,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-03-16 11:53:31,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-03-16 11:53:31,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-03-16 11:53:31,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-03-16 11:53:31,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-03-16 11:53:31,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-03-16 11:53:31,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-03-16 11:53:31,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-03-16 11:53:31,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-03-16 11:53:31,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-03-16 11:53:31,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-03-16 11:53:31,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-03-16 11:53:31,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-03-16 11:53:31,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-03-16 11:53:31,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-03-16 11:53:31,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-03-16 11:53:31,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-03-16 11:53:31,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-03-16 11:53:31,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-03-16 11:53:31,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-03-16 11:53:31,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-03-16 11:53:31,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-03-16 11:53:31,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-03-16 11:53:31,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-03-16 11:53:31,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-03-16 11:53:31,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-03-16 11:53:31,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-03-16 11:53:31,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-03-16 11:53:31,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-03-16 11:53:31,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-03-16 11:53:31,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-03-16 11:53:31,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-03-16 11:53:31,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-03-16 11:53:31,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-03-16 11:53:31,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-03-16 11:53:31,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-03-16 11:53:31,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-03-16 11:53:31,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-03-16 11:53:31,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-03-16 11:53:31,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-03-16 11:53:31,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-03-16 11:53:31,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-03-16 11:53:31,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-03-16 11:53:31,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-03-16 11:53:31,682 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=8.259763985991403E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-16 11:53:31,682 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1157895288:83 , srv1592469582:83 , srv1672116074:83 , srv362987005:83 , srv665286447:83 , srv683618771:83 , srv891566062:83 , srv1001726522:84 , srv1053188406:84 , srv1059793685:84 , srv1085016487:84 , srv1091047118:84 , srv1093511110:84 , srv1097122796:84 , srv1116338148:84 , srv1125411150:84 , srv1203914173:84 , srv1209752207:84 , srv1221097192:84 , srv1249445005:84 , srv127930559:84 , srv1299893502:84 , srv1304728099:84 , srv1313818915:84 , srv131996629:84 , srv1346021856:84 , srv1393213587:84 , srv1452780866:84 , srv1457207707:84 , srv1494764037:84 , srv1512602829:84 , srv1521796083:84 , srv1545361361:84 , srv1563078002:84 , srv1563242430:84 , srv1574680133:84 , srv1594553197:84 , srv1619292252:84 , srv163380578:84 , srv1654967680:84 , srv1686955848:84 , srv1733250273:84 , srv1736087337:84 , srv1742404659:84 , srv1758175228:84 , srv1758493801:84 , srv1762350161:84 , srv1773207715:84 , srv1786314854:84 , srv1795686433:84 , srv182729939:84 , srv1830142214:84 , srv1844062621:84 , srv1872059275:84 , srv1890107802:84 , srv1896600788:84 , srv1992982394:84 , srv1998612417:84 , srv2000223357:84 , srv2004075058:84 , srv2041446057:84 , srv2063788881:84 , srv2068806863:84 , srv2074183981:84 , srv2117013246:84 , srv212750098:84 , srv2139280475:84 , srv27542955:84 , srv296893255:84 , srv376088762:84 , srv390629620:84 , srv4111731:84 , srv418414103:84 , srv430893741:84 , srv457767612:84 , srv457851825:84 , srv470755123:84 , srv47323046:84 , srv516360372:84 , srv525321940:84 , srv535090187:84 , srv539445393:84 , srv629649351:84 , srv651456848:84 , srv671768466:84 , srv697744087:84 , srv729436327:84 , srv78299163:84 , srv818167369:84 , srv821009868:84 , srv836022117:84 , srv906892907:84 , srv1047241842:85 , srv1219359357:85 , srv1290508324:85 , srv1317349341:85 , srv1875388165:85 , srv339433343:85 , srv872680713:85 , srv1406435141:684 }
2024-03-16 11:53:31,718 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=296 (was 287) - SystemLoadAverage LEAK? -, ProcessCount=171 (was 171), AvailableMemoryMB=3426 (was 3943)
2024-03-16 11:53:31,727 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=296, ProcessCount=172, AvailableMemoryMB=3425
2024-03-16 11:53:31,728 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-03-16 11:53:31,728 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2024-03-16 11:53:31,744 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1014787647:84 , srv1142489394:84 , srv1238370025:84 , srv1289412777:84 , srv1332924957:84 , srv1369304535:84 , srv147111937:84 , srv1794745088:84 , srv2021492903:84 , srv202809073:84 , srv2095547917:84 , srv2099612561:84 , srv2105348989:84 , srv251574495:84 , srv255178308:84 , srv355381769:84 , srv37790878:84 , srv47284993:84 , srv512185059:84 , srv537017129:84 , srv552276621:84 , srv58282540:84 , srv63432895:84 , srv685117881:84 , srv687337967:84 , srv866177390:84 , srv882146364:84 , srv961942969:84 , srv96949263:84 , srv585998691:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2024-03-16 11:53:31,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-03-16 11:53:31,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv255178308=14, srv37790878=16, srv1369304535=5, srv585998691=22, srv961942969=28, srv552276621=20, srv1238370025=2, srv1014787647=0, srv2021492903=8, srv685117881=24, srv202809073=9, srv537017129=19, srv355381769=15, srv1289412777=3, srv512185059=18, srv1142489394=1, srv251574495=13, srv58282540=21, srv2099612561=11, srv2105348989=12, srv1794745088=7, srv63432895=23, srv96949263=29, srv47284993=17, srv687337967=25, srv147111937=6, srv1332924957=4, srv2095547917=10, srv882146364=27, srv866177390=26} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-03-16 11:53:31,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-03-16 11:53:31,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-03-16 11:53:31,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-03-16 11:53:31,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-03-16 11:53:31,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-03-16 11:53:31,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-03-16 11:53:31,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-03-16 11:53:31,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-03-16 11:53:31,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-03-16 11:53:31,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-03-16 11:53:31,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-03-16 11:53:31,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-03-16 11:53:31,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-03-16 11:53:31,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-03-16 11:53:31,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-03-16 11:53:31,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-03-16 11:53:31,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-03-16 11:53:31,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-03-16 11:53:31,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-03-16 11:53:31,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-03-16 11:53:31,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-03-16 11:53:31,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-03-16 11:53:31,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-03-16 11:53:31,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-03-16 11:53:31,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-03-16 11:53:31,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-03-16 11:53:31,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-03-16 11:53:31,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-03-16 11:53:31,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-03-16 11:53:31,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-03-16 11:53:31,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-03-16 11:53:31,778 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1593198804736007, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-16 11:53:31,778 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912695055005378, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1593198804736007, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2024-03-16 11:55:31,765 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5495412 different iterations.  Found a solution that moves 1853 regions; Going from a computed imbalance of 0.9912695055005378 to a new imbalance of 4.324994439959734E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6862962962962963, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-16 11:55:31,771 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1014787647:90 , srv1142489394:90 , srv1238370025:90 , srv1289412777:90 , srv1332924957:90 , srv1369304535:90 , srv147111937:90 , srv1794745088:90 , srv2021492903:90 , srv202809073:90 , srv2095547917:90 , srv2099612561:90 , srv2105348989:90 , srv251574495:90 , srv255178308:90 , srv355381769:90 , srv37790878:90 , srv47284993:90 , srv512185059:90 , srv537017129:90 , srv552276621:90 , srv58282540:90 , srv585998691:90 , srv63432895:90 , srv685117881:90 , srv687337967:90 , srv866177390:90 , srv882146364:90 , srv961942969:90 , srv96949263:90 }
2024-03-16 11:55:31,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-03-16 11:55:31,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv255178308=14, srv37790878=16, srv1369304535=5, srv585998691=22, srv961942969=28, srv552276621=20, srv1238370025=2, srv1014787647=0, srv2021492903=8, srv685117881=24, srv202809073=9, srv537017129=19, srv355381769=15, srv1289412777=3, srv512185059=18, srv1142489394=1, srv251574495=13, srv58282540=21, srv2099612561=11, srv2105348989=12, srv1794745088=7, srv63432895=23, srv96949263=29, srv47284993=17, srv687337967=25, srv147111937=6, srv1332924957=4, srv2095547917=10, srv882146364=27, srv866177390=26} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-03-16 11:55:31,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-03-16 11:55:31,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-03-16 11:55:31,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-03-16 11:55:31,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-03-16 11:55:31,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-03-16 11:55:31,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-03-16 11:55:31,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-03-16 11:55:31,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-03-16 11:55:31,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-03-16 11:55:31,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-03-16 11:55:31,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-03-16 11:55:31,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-03-16 11:55:31,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-03-16 11:55:31,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-03-16 11:55:31,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-03-16 11:55:31,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-03-16 11:55:31,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-03-16 11:55:31,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-03-16 11:55:31,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-03-16 11:55:31,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-03-16 11:55:31,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-03-16 11:55:31,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-03-16 11:55:31,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-03-16 11:55:31,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-03-16 11:55:31,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-03-16 11:55:31,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-03-16 11:55:31,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-03-16 11:55:31,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-03-16 11:55:31,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-03-16 11:55:31,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-03-16 11:55:31,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-03-16 11:55:31,788 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-16 11:55:31,788 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1014787647:90 , srv1142489394:90 , srv1238370025:90 , srv1289412777:90 , srv1332924957:90 , srv1369304535:90 , srv147111937:90 , srv1794745088:90 , srv2021492903:90 , srv202809073:90 , srv2095547917:90 , srv2099612561:90 , srv2105348989:90 , srv251574495:90 , srv255178308:90 , srv355381769:90 , srv37790878:90 , srv47284993:90 , srv512185059:90 , srv537017129:90 , srv552276621:90 , srv58282540:90 , srv585998691:90 , srv63432895:90 , srv685117881:90 , srv687337967:90 , srv866177390:90 , srv882146364:90 , srv961942969:90 , srv96949263:90 }
2024-03-16 11:55:31,803 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=181 (was 296), ProcessCount=168 (was 172), AvailableMemoryMB=2544 (was 3425)