2024-01-26 09:53:13,352 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2024-01-26 09:53:13,658 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2024-01-26 09:53:13,772 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2024-01-26 09:53:13,842 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-01-26 09:53:13,850 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-01-26 09:53:13,863 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=268, ProcessCount=167, AvailableMemoryMB=5763
2024-01-26 09:53:13,865 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-01-26 09:53:13,865 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-01-26 09:53:14,007 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1013945090:84 , srv1031931493:84 , srv1046178159:84 , srv1048023265:84 , srv1055932790:84 , srv1075730593:84 , srv1079675115:84 , srv1085233182:84 , srv1113675187:84 , srv1164188193:84 , srv1183251659:84 , srv121490694:84 , srv1222082432:84 , srv1230193216:84 , srv1244903028:84 , srv1288211638:84 , srv1295200031:84 , srv1320833685:84 , srv1388957245:84 , srv1395247268:84 , srv142350950:84 , srv1431245026:84 , srv1434310644:84 , srv1440817235:84 , srv1476412779:84 , srv1489240289:84 , srv1579985826:84 , srv1583863032:84 , srv1597792747:84 , srv1609588180:84 , srv1610226970:84 , srv1618606668:84 , srv1620371410:84 , srv1641479215:84 , srv1661472461:84 , srv1673493308:84 , srv1700587406:84 , srv1714660213:84 , srv1738071524:84 , srv1744217215:84 , srv174932372:84 , srv1769119617:84 , srv1815656891:84 , srv1836144566:84 , srv1906295854:84 , srv1933359232:84 , srv1942392488:84 , srv1946505586:84 , srv1952553590:84 , srv1959463582:84 , srv1975876540:84 , srv19860228:84 , srv1991140319:84 , srv2008719378:84 , srv2016648032:84 , srv2019649024:84 , srv2031938630:84 , srv2042948209:84 , srv2047367625:84 , srv2047487867:84 , srv2067547083:84 , srv2095275402:84 , srv2104654889:84 , srv2113622067:84 , srv211380190:84 , srv219162643:84 , srv224872179:84 , srv230105069:84 , srv255989521:84 , srv277334020:84 , srv303507588:84 , srv311868537:84 , srv32830755:84 , srv380802452:84 , srv396424395:84 , srv411927662:84 , srv429638735:84 , srv433109640:84 , srv44187094:84 , srv453207946:84 , srv497931549:84 , srv500967213:84 , srv538836256:84 , srv564575938:84 , srv637031711:84 , srv649573429:84 , srv694433629:84 , srv726211677:84 , srv74685844:84 , srv801629165:84 , srv809128489:84 , srv809495613:84 , srv82828772:84 , srv874845007:84 , srv892856092:84 , srv928374692:84 , srv930233221:84 , srv930900634:84 , srv993973000:84 , srv313511574:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2024-01-26 09:53:14,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-26 09:53:14,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1700587406=36, srv1610226970=30, srv313511574=72, srv433109640=78, srv19860228=51, srv1113675187=8, srv1476412779=24, srv1222082432=12, srv1431245026=21, srv2008719378=53, srv809495613=92, srv1048023265=3, srv82828772=93, srv1661472461=34, srv411927662=76, srv1744217215=39, srv2047367625=58, srv121490694=11, srv1942392488=46, srv1597792747=28, srv1906295854=44, srv1933359232=45, srv1620371410=32, srv255989521=68, srv1579985826=26, srv726211677=88, srv1946505586=47, srv2047487867=59, srv801629165=90, srv1959463582=49, srv32830755=73, srv874845007=94, srv303507588=70, srv1489240289=25, srv1618606668=31, srv1085233182=7, srv1230193216=13, srv219162643=65, srv1815656891=42, srv1836144566=43, srv74685844=89, srv1769119617=41, srv1440817235=23, srv497931549=81, srv928374692=96, srv1952553590=48, srv809128489=91, srv892856092=95, srv564575938=84, srv1075730593=5, srv1609588180=29, srv1046178159=2, srv142350950=20, srv1738071524=38, srv649573429=86, srv1714660213=37, srv1055932790=4, srv1164188193=9, srv1641479215=33, srv1031931493=1, srv1079675115=6, srv1434310644=22, srv2113622067=63, srv44187094=79, srv694433629=87, srv1320833685=17, srv500967213=82, srv1244903028=14, srv1183251659=10, srv930233221=97, srv453207946=80, srv1673493308=35, srv429638735=77, srv380802452=74, srv311868537=71, srv277334020=69, srv1295200031=16, srv2016648032=54, srv396424395=75, srv1395247268=19, srv538836256=83, srv211380190=64, srv637031711=85, srv2067547083=60, srv2104654889=62, srv174932372=40, srv1583863032=27, srv230105069=67, srv1975876540=50, srv2031938630=56, srv993973000=99, srv1288211638=15, srv1388957245=18, srv2042948209=57, srv1013945090=0, srv224872179=66, srv1991140319=52, srv2095275402=61, srv2019649024=55, srv930900634=98} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-26 09:53:14,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-26 09:53:14,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-26 09:53:14,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-26 09:53:14,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-26 09:53:14,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-26 09:53:14,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-26 09:53:14,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-26 09:53:14,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-26 09:53:14,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-26 09:53:14,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-26 09:53:14,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-26 09:53:14,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-26 09:53:14,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-26 09:53:14,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-26 09:53:14,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-26 09:53:14,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-26 09:53:14,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-26 09:53:14,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-26 09:53:14,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-26 09:53:14,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-26 09:53:14,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-26 09:53:14,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-26 09:53:14,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-26 09:53:14,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-26 09:53:14,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-26 09:53:14,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-26 09:53:14,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-26 09:53:14,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-26 09:53:14,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-26 09:53:14,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-26 09:53:14,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-01-26 09:53:14,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-01-26 09:53:14,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-01-26 09:53:14,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-01-26 09:53:14,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-01-26 09:53:14,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-01-26 09:53:14,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-01-26 09:53:14,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-01-26 09:53:14,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-01-26 09:53:14,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-01-26 09:53:14,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-01-26 09:53:14,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-01-26 09:53:14,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-01-26 09:53:14,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-01-26 09:53:14,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-01-26 09:53:14,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-01-26 09:53:14,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-01-26 09:53:14,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-01-26 09:53:14,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-01-26 09:53:14,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-01-26 09:53:14,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-01-26 09:53:14,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-01-26 09:53:14,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-01-26 09:53:14,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-01-26 09:53:14,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-01-26 09:53:14,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-01-26 09:53:14,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-01-26 09:53:14,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-01-26 09:53:14,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-01-26 09:53:14,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-01-26 09:53:14,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-01-26 09:53:14,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-01-26 09:53:14,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-01-26 09:53:14,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-01-26 09:53:14,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-01-26 09:53:14,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-01-26 09:53:14,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-01-26 09:53:14,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-01-26 09:53:14,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-01-26 09:53:14,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-01-26 09:53:14,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-01-26 09:53:14,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-01-26 09:53:14,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-01-26 09:53:14,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-01-26 09:53:14,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-01-26 09:53:14,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-01-26 09:53:14,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-01-26 09:53:14,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-01-26 09:53:14,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-01-26 09:53:14,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-01-26 09:53:14,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-01-26 09:53:14,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-01-26 09:53:14,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-01-26 09:53:14,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-01-26 09:53:14,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-01-26 09:53:14,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-01-26 09:53:14,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-01-26 09:53:14,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-01-26 09:53:14,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-01-26 09:53:14,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-01-26 09:53:14,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-01-26 09:53:14,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-01-26 09:53:14,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-01-26 09:53:14,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-01-26 09:53:14,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-01-26 09:53:14,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-01-26 09:53:14,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-01-26 09:53:14,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-01-26 09:53:14,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-01-26 09:53:14,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-01-26 09:53:14,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-01-26 09:53:14,337 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-26 09:53:14,338 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-01-26 09:53:14,338 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-01-26 09:53:14,874 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 727 ms to try 5000 different iterations.  Found a solution that moves 4690 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17897964300555624. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04262259227500045, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05647521629986664, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5211111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04262259227500045, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17408333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24175, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-26 09:53:14,885 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv928374692:80 , srv1288211638:81 , srv1031931493:82 , srv1079675115:82 , srv1641479215:82 , srv1836144566:82 , srv1933359232:83 , srv1952553590:83 , srv2047367625:83 , srv277334020:83 , srv311868537:83 , srv44187094:83 , srv930233221:83 , srv1048023265:84 , srv1183251659:84 , srv121490694:84 , srv1395247268:84 , srv142350950:84 , srv1609588180:84 , srv1618606668:84 , srv1661472461:84 , srv1744217215:84 , srv500967213:84 , srv637031711:84 , srv1085233182:85 , srv1113675187:85 , srv1431245026:85 , srv1620371410:85 , srv1673493308:85 , srv1700587406:85 , srv1714660213:85 , srv1959463582:85 , srv19860228:85 , srv2019649024:85 , srv219162643:85 , srv255989521:85 , srv396424395:85 , srv429638735:85 , srv538836256:85 , srv801629165:85 , srv82828772:85 , srv1489240289:86 , srv1579985826:86 , srv1610226970:86 , srv1738071524:86 , srv1942392488:86 , srv2008719378:86 , srv2031938630:86 , srv211380190:86 , srv32830755:86 , srv453207946:86 , srv564575938:86 , srv809128489:86 , srv809495613:86 , srv930900634:86 , srv993973000:86 , srv1164188193:87 , srv1244903028:87 , srv1440817235:87 , srv174932372:87 , srv1906295854:87 , srv1991140319:87 , srv2042948209:87 , srv2104654889:87 , srv2113622067:87 , srv224872179:87 , srv433109640:87 , srv694433629:87 , srv74685844:87 , srv1013945090:88 , srv1295200031:88 , srv1388957245:88 , srv1597792747:88 , srv1815656891:88 , srv1975876540:88 , srv2067547083:88 , srv2095275402:88 , srv230105069:88 , srv303507588:88 , srv649573429:88 , srv726211677:88 , srv874845007:88 , srv1055932790:89 , srv1075730593:89 , srv1476412779:89 , srv1583863032:89 , srv1769119617:89 , srv1946505586:89 , srv2016648032:89 , srv2047487867:89 , srv411927662:89 , srv497931549:89 , srv892856092:89 , srv1046178159:90 , srv1222082432:90 , srv1320833685:91 , srv1230193216:92 , srv1434310644:92 , srv380802452:93 , srv313511574:469 }
2024-01-26 09:53:14,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-26 09:53:14,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1700587406=36, srv1610226970=30, srv313511574=72, srv433109640=78, srv19860228=51, srv1113675187=8, srv1476412779=24, srv1222082432=12, srv1431245026=21, srv2008719378=53, srv809495613=92, srv1048023265=3, srv82828772=93, srv1661472461=34, srv411927662=76, srv1744217215=39, srv2047367625=58, srv121490694=11, srv1942392488=46, srv1597792747=28, srv1906295854=44, srv1933359232=45, srv1620371410=32, srv255989521=68, srv1579985826=26, srv726211677=88, srv1946505586=47, srv2047487867=59, srv801629165=90, srv1959463582=49, srv32830755=73, srv874845007=94, srv303507588=70, srv1489240289=25, srv1618606668=31, srv1085233182=7, srv1230193216=13, srv219162643=65, srv1815656891=42, srv1836144566=43, srv74685844=89, srv1769119617=41, srv1440817235=23, srv497931549=81, srv928374692=96, srv1952553590=48, srv809128489=91, srv892856092=95, srv564575938=84, srv1075730593=5, srv1609588180=29, srv1046178159=2, srv142350950=20, srv1738071524=38, srv649573429=86, srv1714660213=37, srv1055932790=4, srv1164188193=9, srv1641479215=33, srv1031931493=1, srv1079675115=6, srv1434310644=22, srv2113622067=63, srv44187094=79, srv694433629=87, srv1320833685=17, srv500967213=82, srv1244903028=14, srv1183251659=10, srv930233221=97, srv453207946=80, srv1673493308=35, srv429638735=77, srv380802452=74, srv311868537=71, srv277334020=69, srv1295200031=16, srv2016648032=54, srv396424395=75, srv1395247268=19, srv538836256=83, srv211380190=64, srv637031711=85, srv2067547083=60, srv2104654889=62, srv174932372=40, srv1583863032=27, srv230105069=67, srv1975876540=50, srv2031938630=56, srv993973000=99, srv1288211638=15, srv1388957245=18, srv2042948209=57, srv1013945090=0, srv224872179=66, srv1991140319=52, srv2095275402=61, srv2019649024=55, srv930900634=98} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-26 09:53:14,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-26 09:53:14,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-26 09:53:14,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-26 09:53:14,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-26 09:53:14,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-26 09:53:14,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-26 09:53:14,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-26 09:53:14,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-26 09:53:14,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-26 09:53:14,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-26 09:53:14,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-26 09:53:14,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-26 09:53:14,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-26 09:53:14,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-26 09:53:14,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-26 09:53:14,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-26 09:53:14,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-26 09:53:14,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-26 09:53:14,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-26 09:53:14,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-26 09:53:14,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-26 09:53:14,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-26 09:53:14,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-26 09:53:14,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-26 09:53:14,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-26 09:53:14,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-26 09:53:14,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-26 09:53:14,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-26 09:53:14,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-26 09:53:14,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-26 09:53:14,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-01-26 09:53:14,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-01-26 09:53:14,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-01-26 09:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-01-26 09:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-01-26 09:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-01-26 09:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-01-26 09:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-01-26 09:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-01-26 09:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-01-26 09:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-01-26 09:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-01-26 09:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-01-26 09:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-01-26 09:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-01-26 09:53:14,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-01-26 09:53:14,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-01-26 09:53:14,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-01-26 09:53:14,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-01-26 09:53:14,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-01-26 09:53:14,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-01-26 09:53:14,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-01-26 09:53:14,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-01-26 09:53:14,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-01-26 09:53:14,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-01-26 09:53:14,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-01-26 09:53:14,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-01-26 09:53:14,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-01-26 09:53:14,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-01-26 09:53:14,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-01-26 09:53:14,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-01-26 09:53:14,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-01-26 09:53:14,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-01-26 09:53:14,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-01-26 09:53:14,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-01-26 09:53:14,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-01-26 09:53:14,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-01-26 09:53:14,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-01-26 09:53:14,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-01-26 09:53:14,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-01-26 09:53:14,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-01-26 09:53:14,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-01-26 09:53:14,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-01-26 09:53:14,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-01-26 09:53:14,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-01-26 09:53:14,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-01-26 09:53:14,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-01-26 09:53:14,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-01-26 09:53:14,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-01-26 09:53:14,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-01-26 09:53:14,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-01-26 09:53:14,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-01-26 09:53:14,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-01-26 09:53:14,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-01-26 09:53:14,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-01-26 09:53:14,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-01-26 09:53:14,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-01-26 09:53:14,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-01-26 09:53:14,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-01-26 09:53:14,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-01-26 09:53:14,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-01-26 09:53:14,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-01-26 09:53:14,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-01-26 09:53:14,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-01-26 09:53:14,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-01-26 09:53:14,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-01-26 09:53:14,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-01-26 09:53:14,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-01-26 09:53:14,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-01-26 09:53:14,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-01-26 09:53:14,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-01-26 09:53:15,078 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04262259227500045, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05647521629986664, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04262259227500045, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17408333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24175, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-26 09:53:15,078 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-01-26 09:53:15,078 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17894680292365112, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04262259227500045, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05647521629986664, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04262259227500045, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17408333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24175, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-01-26 09:53:15,429 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 485 ms to try 5000 different iterations.  Found a solution that moves 2629 regions; Going from a computed imbalance of 0.17894680292365112 to a new imbalance of 0.015537610789166649. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023051853582958914); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.036320681700539696, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2921111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023051853582958914); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01225); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04683333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-26 09:53:15,443 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1320833685:78 , srv1230193216:80 , srv1434310644:80 , srv380802452:80 , srv497931549:80 , srv1583863032:81 , srv411927662:81 , srv1046178159:82 , srv1055932790:82 , srv1295200031:82 , srv1388957245:82 , srv1769119617:82 , srv1946505586:82 , srv874845007:82 , srv1222082432:83 , srv1579985826:83 , srv2016648032:83 , srv649573429:83 , srv1013945090:84 , srv1075730593:84 , srv1476412779:84 , srv1975876540:84 , srv1991140319:84 , srv2047487867:84 , srv2104654889:84 , srv303507588:84 , srv453207946:84 , srv694433629:84 , srv809495613:84 , srv892856092:84 , srv1085233182:85 , srv1164188193:85 , srv1244903028:85 , srv1597792747:85 , srv1738071524:85 , srv174932372:85 , srv1815656891:85 , srv1906295854:85 , srv1942392488:85 , srv1959463582:85 , srv2067547083:85 , srv2113622067:85 , srv224872179:85 , srv74685844:85 , srv142350950:86 , srv1431245026:86 , srv1489240289:86 , srv1610226970:86 , srv1620371410:86 , srv1714660213:86 , srv2031938630:86 , srv2042948209:86 , srv2095275402:86 , srv32830755:86 , srv433109640:86 , srv726211677:86 , srv809128489:86 , srv993973000:86 , srv1048023265:87 , srv1113675187:87 , srv1395247268:87 , srv1440817235:87 , srv1609588180:87 , srv1618606668:87 , srv1673493308:87 , srv1933359232:87 , srv19860228:87 , srv2019649024:87 , srv211380190:87 , srv219162643:87 , srv230105069:87 , srv255989521:87 , srv396424395:87 , srv429638735:87 , srv44187094:87 , srv564575938:87 , srv930900634:87 , srv1183251659:88 , srv121490694:88 , srv1700587406:88 , srv1744217215:88 , srv2008719378:88 , srv538836256:88 , srv637031711:88 , srv82828772:88 , srv1031931493:89 , srv311868537:89 , srv500967213:89 , srv1079675115:90 , srv1661472461:90 , srv2047367625:90 , srv801629165:90 , srv1288211638:91 , srv1641479215:91 , srv1836144566:91 , srv1952553590:91 , srv277334020:91 , srv930233221:91 , srv928374692:92 , srv313511574:510 }
2024-01-26 09:53:15,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-26 09:53:15,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1700587406=36, srv1610226970=30, srv313511574=72, srv433109640=78, srv19860228=51, srv1113675187=8, srv1476412779=24, srv1222082432=12, srv1431245026=21, srv2008719378=53, srv809495613=92, srv1048023265=3, srv82828772=93, srv1661472461=34, srv411927662=76, srv1744217215=39, srv2047367625=58, srv121490694=11, srv1942392488=46, srv1597792747=28, srv1906295854=44, srv1933359232=45, srv1620371410=32, srv255989521=68, srv1579985826=26, srv726211677=88, srv1946505586=47, srv2047487867=59, srv801629165=90, srv1959463582=49, srv32830755=73, srv874845007=94, srv303507588=70, srv1489240289=25, srv1618606668=31, srv1085233182=7, srv1230193216=13, srv219162643=65, srv1815656891=42, srv1836144566=43, srv74685844=89, srv1769119617=41, srv1440817235=23, srv497931549=81, srv928374692=96, srv1952553590=48, srv809128489=91, srv892856092=95, srv564575938=84, srv1075730593=5, srv1609588180=29, srv1046178159=2, srv142350950=20, srv1738071524=38, srv649573429=86, srv1714660213=37, srv1055932790=4, srv1164188193=9, srv1641479215=33, srv1031931493=1, srv1079675115=6, srv1434310644=22, srv2113622067=63, srv44187094=79, srv694433629=87, srv1320833685=17, srv500967213=82, srv1244903028=14, srv1183251659=10, srv930233221=97, srv453207946=80, srv1673493308=35, srv429638735=77, srv380802452=74, srv311868537=71, srv277334020=69, srv1295200031=16, srv2016648032=54, srv396424395=75, srv1395247268=19, srv538836256=83, srv211380190=64, srv637031711=85, srv2067547083=60, srv2104654889=62, srv174932372=40, srv1583863032=27, srv230105069=67, srv1975876540=50, srv2031938630=56, srv993973000=99, srv1288211638=15, srv1388957245=18, srv2042948209=57, srv1013945090=0, srv224872179=66, srv1991140319=52, srv2095275402=61, srv2019649024=55, srv930900634=98} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-26 09:53:15,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-26 09:53:15,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-26 09:53:15,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-26 09:53:15,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-26 09:53:15,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-26 09:53:15,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-26 09:53:15,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-26 09:53:15,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-26 09:53:15,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-26 09:53:15,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-26 09:53:15,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-26 09:53:15,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-26 09:53:15,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-26 09:53:15,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-26 09:53:15,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-26 09:53:15,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-26 09:53:15,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-26 09:53:15,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-26 09:53:15,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-26 09:53:15,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-26 09:53:15,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-26 09:53:15,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-26 09:53:15,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-26 09:53:15,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-26 09:53:15,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-26 09:53:15,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-26 09:53:15,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-26 09:53:15,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-26 09:53:15,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-26 09:53:15,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-26 09:53:15,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-01-26 09:53:15,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-01-26 09:53:15,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-01-26 09:53:15,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-01-26 09:53:15,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-01-26 09:53:15,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-01-26 09:53:15,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-01-26 09:53:15,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-01-26 09:53:15,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-01-26 09:53:15,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-01-26 09:53:15,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-01-26 09:53:15,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-01-26 09:53:15,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-01-26 09:53:15,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-01-26 09:53:15,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-01-26 09:53:15,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-01-26 09:53:15,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-01-26 09:53:15,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-01-26 09:53:15,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-01-26 09:53:15,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-01-26 09:53:15,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-01-26 09:53:15,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-01-26 09:53:15,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-01-26 09:53:15,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-01-26 09:53:15,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-01-26 09:53:15,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-01-26 09:53:15,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-01-26 09:53:15,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-01-26 09:53:15,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-01-26 09:53:15,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-01-26 09:53:15,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-01-26 09:53:15,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-01-26 09:53:15,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-01-26 09:53:15,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-01-26 09:53:15,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-01-26 09:53:15,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-01-26 09:53:15,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-01-26 09:53:15,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-01-26 09:53:15,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-01-26 09:53:15,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-01-26 09:53:15,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-01-26 09:53:15,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-01-26 09:53:15,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-01-26 09:53:15,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-01-26 09:53:15,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-01-26 09:53:15,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-01-26 09:53:15,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-01-26 09:53:15,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-01-26 09:53:15,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-01-26 09:53:15,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-01-26 09:53:15,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-01-26 09:53:15,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-01-26 09:53:15,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-01-26 09:53:15,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-01-26 09:53:15,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-01-26 09:53:15,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-01-26 09:53:15,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-01-26 09:53:15,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-01-26 09:53:15,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-01-26 09:53:15,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-01-26 09:53:15,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-01-26 09:53:15,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-01-26 09:53:15,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-01-26 09:53:15,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-01-26 09:53:15,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-01-26 09:53:15,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-01-26 09:53:15,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-01-26 09:53:15,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-01-26 09:53:15,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-01-26 09:53:15,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-01-26 09:53:15,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-01-26 09:53:15,689 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023051853582958914); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.036320681700539696, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023051853582958914); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01225); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04683333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-26 09:53:15,689 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-01-26 09:53:15,690 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.015519202137710653, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023051853582958914); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.036320681700539696, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023051853582958914); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01225); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04683333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-01-26 09:53:16,208 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 706 ms to try 5000 different iterations.  Found a solution that moves 991 regions; Going from a computed imbalance of 0.015519202137710653 to a new imbalance of 5.639939198370576E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008804311428960357); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.014331336482938547); MoveCostFunction : (multiplier=7.0, imbalance=0.11011111111111112, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.008804311428960357); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0033333333333333335); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-26 09:53:16,211 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv277334020:82 , srv1075730593:83 , srv1440817235:83 , srv1700587406:83 , srv1815656891:83 , srv1952553590:83 , srv2008719378:83 , srv2095275402:83 , srv211380190:83 , srv230105069:83 , srv801629165:83 , srv892856092:83 , srv930900634:83 , srv1222082432:84 , srv1476412779:84 , srv1597792747:84 , srv1661472461:84 , srv174932372:84 , srv2042948209:84 , srv2047487867:84 , srv2067547083:84 , srv2113622067:84 , srv538836256:84 , srv1031931493:85 , srv1079675115:85 , srv1113675187:85 , srv1183251659:85 , srv121490694:85 , srv1230193216:85 , srv1244903028:85 , srv1288211638:85 , srv1395247268:85 , srv1431245026:85 , srv1610226970:85 , srv1618606668:85 , srv1641479215:85 , srv1673493308:85 , srv1714660213:85 , srv1744217215:85 , srv1769119617:85 , srv1906295854:85 , srv1942392488:85 , srv1975876540:85 , srv1991140319:85 , srv2047367625:85 , srv219162643:85 , srv224872179:85 , srv255989521:85 , srv303507588:85 , srv311868537:85 , srv32830755:85 , srv380802452:85 , srv429638735:85 , srv433109640:85 , srv500967213:85 , srv564575938:85 , srv649573429:85 , srv726211677:85 , srv74685844:85 , srv82828772:85 , srv930233221:85 , srv1013945090:86 , srv1048023265:86 , srv1055932790:86 , srv1164188193:86 , srv1388957245:86 , srv1434310644:86 , srv1489240289:86 , srv1620371410:86 , srv1738071524:86 , srv1836144566:86 , srv1946505586:86 , srv19860228:86 , srv2016648032:86 , srv2019649024:86 , srv2031938630:86 , srv2104654889:86 , srv396424395:86 , srv44187094:86 , srv637031711:86 , srv809128489:86 , srv928374692:86 , srv993973000:86 , srv1046178159:87 , srv1085233182:87 , srv1609588180:87 , srv1933359232:87 , srv1959463582:87 , srv411927662:87 , srv694433629:87 , srv809495613:87 , srv874845007:87 , srv1295200031:88 , srv1320833685:88 , srv1579985826:88 , srv142350950:89 , srv1583863032:89 , srv453207946:89 , srv497931549:89 , srv313511574:557 }
2024-01-26 09:53:16,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-26 09:53:16,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1700587406=36, srv1610226970=30, srv313511574=72, srv433109640=78, srv19860228=51, srv1113675187=8, srv1476412779=24, srv1222082432=12, srv1431245026=21, srv2008719378=53, srv809495613=92, srv1048023265=3, srv82828772=93, srv1661472461=34, srv411927662=76, srv1744217215=39, srv2047367625=58, srv121490694=11, srv1942392488=46, srv1597792747=28, srv1906295854=44, srv1933359232=45, srv1620371410=32, srv255989521=68, srv1579985826=26, srv726211677=88, srv1946505586=47, srv2047487867=59, srv801629165=90, srv1959463582=49, srv32830755=73, srv874845007=94, srv303507588=70, srv1489240289=25, srv1618606668=31, srv1085233182=7, srv1230193216=13, srv219162643=65, srv1815656891=42, srv1836144566=43, srv74685844=89, srv1769119617=41, srv1440817235=23, srv497931549=81, srv928374692=96, srv1952553590=48, srv809128489=91, srv892856092=95, srv564575938=84, srv1075730593=5, srv1609588180=29, srv1046178159=2, srv142350950=20, srv1738071524=38, srv649573429=86, srv1714660213=37, srv1055932790=4, srv1164188193=9, srv1641479215=33, srv1031931493=1, srv1079675115=6, srv1434310644=22, srv2113622067=63, srv44187094=79, srv694433629=87, srv1320833685=17, srv500967213=82, srv1244903028=14, srv1183251659=10, srv930233221=97, srv453207946=80, srv1673493308=35, srv429638735=77, srv380802452=74, srv311868537=71, srv277334020=69, srv1295200031=16, srv2016648032=54, srv396424395=75, srv1395247268=19, srv538836256=83, srv211380190=64, srv637031711=85, srv2067547083=60, srv2104654889=62, srv174932372=40, srv1583863032=27, srv230105069=67, srv1975876540=50, srv2031938630=56, srv993973000=99, srv1288211638=15, srv1388957245=18, srv2042948209=57, srv1013945090=0, srv224872179=66, srv1991140319=52, srv2095275402=61, srv2019649024=55, srv930900634=98} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-26 09:53:16,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-26 09:53:16,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-26 09:53:16,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-26 09:53:16,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-26 09:53:16,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-26 09:53:16,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-26 09:53:16,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-26 09:53:16,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-26 09:53:16,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-26 09:53:16,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-26 09:53:16,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-26 09:53:16,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-26 09:53:16,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-26 09:53:16,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-26 09:53:16,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-26 09:53:16,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-26 09:53:16,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-26 09:53:16,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-26 09:53:16,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-26 09:53:16,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-26 09:53:16,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-26 09:53:16,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-26 09:53:16,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-26 09:53:16,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-26 09:53:16,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-26 09:53:16,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-26 09:53:16,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-26 09:53:16,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-26 09:53:16,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-26 09:53:16,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-26 09:53:16,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-01-26 09:53:16,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-01-26 09:53:16,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-01-26 09:53:16,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-01-26 09:53:16,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-01-26 09:53:16,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-01-26 09:53:16,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-01-26 09:53:16,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-01-26 09:53:16,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-01-26 09:53:16,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-01-26 09:53:16,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-01-26 09:53:16,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-01-26 09:53:16,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-01-26 09:53:16,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-01-26 09:53:16,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-01-26 09:53:16,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-01-26 09:53:16,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-01-26 09:53:16,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-01-26 09:53:16,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-01-26 09:53:16,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-01-26 09:53:16,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-01-26 09:53:16,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-01-26 09:53:16,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-01-26 09:53:16,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-01-26 09:53:16,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-01-26 09:53:16,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-01-26 09:53:16,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-01-26 09:53:16,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-01-26 09:53:16,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-01-26 09:53:16,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-01-26 09:53:16,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-01-26 09:53:16,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-01-26 09:53:16,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-01-26 09:53:16,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-01-26 09:53:16,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-01-26 09:53:16,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-01-26 09:53:16,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-01-26 09:53:16,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-01-26 09:53:16,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-01-26 09:53:16,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-01-26 09:53:16,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-01-26 09:53:16,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-01-26 09:53:16,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-01-26 09:53:16,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-01-26 09:53:16,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-01-26 09:53:16,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-01-26 09:53:16,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-01-26 09:53:16,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-01-26 09:53:16,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-01-26 09:53:16,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-01-26 09:53:16,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-01-26 09:53:16,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-01-26 09:53:16,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-01-26 09:53:16,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-01-26 09:53:16,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-01-26 09:53:16,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-01-26 09:53:16,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-01-26 09:53:16,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-01-26 09:53:16,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-01-26 09:53:16,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-01-26 09:53:16,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-01-26 09:53:16,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-01-26 09:53:16,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-01-26 09:53:16,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-01-26 09:53:16,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-01-26 09:53:16,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-01-26 09:53:16,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-01-26 09:53:16,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-01-26 09:53:16,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-01-26 09:53:16,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-01-26 09:53:16,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-01-26 09:53:16,377 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008804311428960357); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.014331336482938547); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.008804311428960357); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0033333333333333335); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-26 09:53:16,377 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-01-26 09:53:16,378 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=5.570547895240515E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008804311428960357); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.014331336482938547); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.008804311428960357); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0033333333333333335); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-01-26 09:53:16,705 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 444 ms to try 5000 different iterations.  Found a solution that moves 266 regions; Going from a computed imbalance of 5.570547895240515E-4 to a new imbalance of 4.307134416464617E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.029555555555555557, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-26 09:53:16,706 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv726211677:82 , srv1046178159:83 , srv1295200031:83 , srv142350950:83 , srv1489240289:83 , srv2031938630:83 , srv453207946:83 , srv930233221:83 , srv1013945090:84 , srv1164188193:84 , srv1583863032:84 , srv1609588180:84 , srv1641479215:84 , srv1661472461:84 , srv1836144566:84 , srv19860228:84 , srv2016648032:84 , srv2019649024:84 , srv2047367625:84 , srv230105069:84 , srv380802452:84 , srv396424395:84 , srv411927662:84 , srv433109640:84 , srv497931549:84 , srv500967213:84 , srv564575938:84 , srv637031711:84 , srv694433629:84 , srv801629165:84 , srv809495613:84 , srv82828772:84 , srv928374692:84 , srv993973000:84 , srv1048023265:85 , srv1055932790:85 , srv1079675115:85 , srv1085233182:85 , srv1113675187:85 , srv1183251659:85 , srv121490694:85 , srv1222082432:85 , srv1230193216:85 , srv1244903028:85 , srv1288211638:85 , srv1320833685:85 , srv1434310644:85 , srv1440817235:85 , srv1476412779:85 , srv1579985826:85 , srv1597792747:85 , srv1610226970:85 , srv1618606668:85 , srv1620371410:85 , srv1673493308:85 , srv1744217215:85 , srv1769119617:85 , srv1906295854:85 , srv1933359232:85 , srv1946505586:85 , srv1952553590:85 , srv1959463582:85 , srv1975876540:85 , srv2008719378:85 , srv2042948209:85 , srv2047487867:85 , srv2067547083:85 , srv2095275402:85 , srv2104654889:85 , srv219162643:85 , srv303507588:85 , srv311868537:85 , srv32830755:85 , srv429638735:85 , srv538836256:85 , srv74685844:85 , srv809128489:85 , srv874845007:85 , srv1031931493:86 , srv1075730593:86 , srv1388957245:86 , srv1395247268:86 , srv1431245026:86 , srv1700587406:86 , srv1714660213:86 , srv1738071524:86 , srv174932372:86 , srv1815656891:86 , srv1942392488:86 , srv1991140319:86 , srv211380190:86 , srv224872179:86 , srv255989521:86 , srv277334020:86 , srv44187094:86 , srv649573429:86 , srv892856092:86 , srv930900634:86 , srv2113622067:87 , srv313511574:606 }
2024-01-26 09:53:16,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-26 09:53:16,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1700587406=36, srv1610226970=30, srv313511574=72, srv433109640=78, srv19860228=51, srv1113675187=8, srv1476412779=24, srv1222082432=12, srv1431245026=21, srv2008719378=53, srv809495613=92, srv1048023265=3, srv82828772=93, srv1661472461=34, srv411927662=76, srv1744217215=39, srv2047367625=58, srv121490694=11, srv1942392488=46, srv1597792747=28, srv1906295854=44, srv1933359232=45, srv1620371410=32, srv255989521=68, srv1579985826=26, srv726211677=88, srv1946505586=47, srv2047487867=59, srv801629165=90, srv1959463582=49, srv32830755=73, srv874845007=94, srv303507588=70, srv1489240289=25, srv1618606668=31, srv1085233182=7, srv1230193216=13, srv219162643=65, srv1815656891=42, srv1836144566=43, srv74685844=89, srv1769119617=41, srv1440817235=23, srv497931549=81, srv928374692=96, srv1952553590=48, srv809128489=91, srv892856092=95, srv564575938=84, srv1075730593=5, srv1609588180=29, srv1046178159=2, srv142350950=20, srv1738071524=38, srv649573429=86, srv1714660213=37, srv1055932790=4, srv1164188193=9, srv1641479215=33, srv1031931493=1, srv1079675115=6, srv1434310644=22, srv2113622067=63, srv44187094=79, srv694433629=87, srv1320833685=17, srv500967213=82, srv1244903028=14, srv1183251659=10, srv930233221=97, srv453207946=80, srv1673493308=35, srv429638735=77, srv380802452=74, srv311868537=71, srv277334020=69, srv1295200031=16, srv2016648032=54, srv396424395=75, srv1395247268=19, srv538836256=83, srv211380190=64, srv637031711=85, srv2067547083=60, srv2104654889=62, srv174932372=40, srv1583863032=27, srv230105069=67, srv1975876540=50, srv2031938630=56, srv993973000=99, srv1288211638=15, srv1388957245=18, srv2042948209=57, srv1013945090=0, srv224872179=66, srv1991140319=52, srv2095275402=61, srv2019649024=55, srv930900634=98} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-26 09:53:16,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-26 09:53:16,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-26 09:53:16,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-26 09:53:16,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-26 09:53:16,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-26 09:53:16,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-26 09:53:16,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-26 09:53:16,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-26 09:53:16,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-26 09:53:16,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-26 09:53:16,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-26 09:53:16,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-26 09:53:16,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-26 09:53:16,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-26 09:53:16,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-26 09:53:16,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-26 09:53:16,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-26 09:53:16,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-26 09:53:16,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-26 09:53:16,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-26 09:53:16,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-26 09:53:16,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-26 09:53:16,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-26 09:53:16,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-26 09:53:16,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-26 09:53:16,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-26 09:53:16,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-26 09:53:16,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-26 09:53:16,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-26 09:53:16,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-26 09:53:16,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-01-26 09:53:16,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-01-26 09:53:16,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-01-26 09:53:16,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-01-26 09:53:16,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-01-26 09:53:16,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-01-26 09:53:16,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-01-26 09:53:16,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-01-26 09:53:16,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-01-26 09:53:16,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-01-26 09:53:16,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-01-26 09:53:16,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-01-26 09:53:16,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-01-26 09:53:16,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-01-26 09:53:16,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-01-26 09:53:16,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-01-26 09:53:16,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-01-26 09:53:16,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-01-26 09:53:16,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-01-26 09:53:16,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-01-26 09:53:16,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-01-26 09:53:16,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-01-26 09:53:16,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-01-26 09:53:16,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-01-26 09:53:16,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-01-26 09:53:16,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-01-26 09:53:16,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-01-26 09:53:16,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-01-26 09:53:16,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-01-26 09:53:16,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-01-26 09:53:16,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-01-26 09:53:16,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-01-26 09:53:16,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-01-26 09:53:16,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-01-26 09:53:16,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-01-26 09:53:16,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-01-26 09:53:16,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-01-26 09:53:16,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-01-26 09:53:16,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-01-26 09:53:16,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-01-26 09:53:16,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-01-26 09:53:16,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-01-26 09:53:16,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-01-26 09:53:16,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-01-26 09:53:16,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-01-26 09:53:16,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-01-26 09:53:16,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-01-26 09:53:16,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-01-26 09:53:16,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-01-26 09:53:16,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-01-26 09:53:16,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-01-26 09:53:16,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-01-26 09:53:16,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-01-26 09:53:16,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-01-26 09:53:16,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-01-26 09:53:16,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-01-26 09:53:16,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-01-26 09:53:16,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-01-26 09:53:16,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-01-26 09:53:16,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-01-26 09:53:16,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-01-26 09:53:16,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-01-26 09:53:16,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-01-26 09:53:16,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-01-26 09:53:16,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-01-26 09:53:16,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-01-26 09:53:16,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-01-26 09:53:16,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-01-26 09:53:16,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-01-26 09:53:16,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-01-26 09:53:16,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-01-26 09:53:16,841 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-26 09:53:16,842 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-01-26 09:53:16,842 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=4.120877235510064E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-01-26 09:53:17,111 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 376 ms to try 5000 different iterations.  Found a solution that moves 33 regions; Going from a computed imbalance of 4.120877235510064E-5 to a new imbalance of 6.629056611047903E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014213381090374027); MoveCostFunction : (multiplier=7.0, imbalance=0.0036666666666666666); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-26 09:53:17,111 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1434310644:83 , srv1583863032:83 , srv1738071524:83 , srv1836144566:83 , srv2113622067:83 , srv224872179:83 , srv255989521:83 , srv809128489:83 , srv1013945090:84 , srv1031931493:84 , srv1046178159:84 , srv1048023265:84 , srv1055932790:84 , srv1075730593:84 , srv1079675115:84 , srv1085233182:84 , srv1113675187:84 , srv1164188193:84 , srv1183251659:84 , srv121490694:84 , srv1222082432:84 , srv1230193216:84 , srv1244903028:84 , srv1288211638:84 , srv1320833685:84 , srv1388957245:84 , srv1395247268:84 , srv142350950:84 , srv1431245026:84 , srv1440817235:84 , srv1476412779:84 , srv1579985826:84 , srv1597792747:84 , srv1609588180:84 , srv1610226970:84 , srv1620371410:84 , srv1641479215:84 , srv1661472461:84 , srv1673493308:84 , srv1700587406:84 , srv1714660213:84 , srv1744217215:84 , srv174932372:84 , srv1815656891:84 , srv1906295854:84 , srv1933359232:84 , srv1942392488:84 , srv1946505586:84 , srv1952553590:84 , srv1959463582:84 , srv1975876540:84 , srv19860228:84 , srv1991140319:84 , srv2008719378:84 , srv2016648032:84 , srv2019649024:84 , srv2042948209:84 , srv2047367625:84 , srv2047487867:84 , srv2067547083:84 , srv2095275402:84 , srv2104654889:84 , srv211380190:84 , srv219162643:84 , srv230105069:84 , srv277334020:84 , srv303507588:84 , srv311868537:84 , srv32830755:84 , srv380802452:84 , srv396424395:84 , srv429638735:84 , srv433109640:84 , srv44187094:84 , srv453207946:84 , srv497931549:84 , srv500967213:84 , srv538836256:84 , srv564575938:84 , srv637031711:84 , srv649573429:84 , srv694433629:84 , srv74685844:84 , srv801629165:84 , srv82828772:84 , srv874845007:84 , srv892856092:84 , srv928374692:84 , srv930233221:84 , srv930900634:84 , srv993973000:84 , srv1295200031:85 , srv1489240289:85 , srv1618606668:85 , srv1769119617:85 , srv2031938630:85 , srv411927662:85 , srv726211677:85 , srv809495613:85 , srv313511574:684 }
2024-01-26 09:53:17,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-26 09:53:17,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1700587406=36, srv1610226970=30, srv313511574=72, srv433109640=78, srv19860228=51, srv1113675187=8, srv1476412779=24, srv1222082432=12, srv1431245026=21, srv2008719378=53, srv809495613=92, srv1048023265=3, srv82828772=93, srv1661472461=34, srv411927662=76, srv1744217215=39, srv2047367625=58, srv121490694=11, srv1942392488=46, srv1597792747=28, srv1906295854=44, srv1933359232=45, srv1620371410=32, srv255989521=68, srv1579985826=26, srv726211677=88, srv1946505586=47, srv2047487867=59, srv801629165=90, srv1959463582=49, srv32830755=73, srv874845007=94, srv303507588=70, srv1489240289=25, srv1618606668=31, srv1085233182=7, srv1230193216=13, srv219162643=65, srv1815656891=42, srv1836144566=43, srv74685844=89, srv1769119617=41, srv1440817235=23, srv497931549=81, srv928374692=96, srv1952553590=48, srv809128489=91, srv892856092=95, srv564575938=84, srv1075730593=5, srv1609588180=29, srv1046178159=2, srv142350950=20, srv1738071524=38, srv649573429=86, srv1714660213=37, srv1055932790=4, srv1164188193=9, srv1641479215=33, srv1031931493=1, srv1079675115=6, srv1434310644=22, srv2113622067=63, srv44187094=79, srv694433629=87, srv1320833685=17, srv500967213=82, srv1244903028=14, srv1183251659=10, srv930233221=97, srv453207946=80, srv1673493308=35, srv429638735=77, srv380802452=74, srv311868537=71, srv277334020=69, srv1295200031=16, srv2016648032=54, srv396424395=75, srv1395247268=19, srv538836256=83, srv211380190=64, srv637031711=85, srv2067547083=60, srv2104654889=62, srv174932372=40, srv1583863032=27, srv230105069=67, srv1975876540=50, srv2031938630=56, srv993973000=99, srv1288211638=15, srv1388957245=18, srv2042948209=57, srv1013945090=0, srv224872179=66, srv1991140319=52, srv2095275402=61, srv2019649024=55, srv930900634=98} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-26 09:53:17,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-26 09:53:17,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-26 09:53:17,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-26 09:53:17,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-26 09:53:17,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-26 09:53:17,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-26 09:53:17,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-26 09:53:17,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-26 09:53:17,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-26 09:53:17,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-26 09:53:17,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-26 09:53:17,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-26 09:53:17,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-26 09:53:17,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-26 09:53:17,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-26 09:53:17,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-26 09:53:17,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-26 09:53:17,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-26 09:53:17,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-26 09:53:17,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-26 09:53:17,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-26 09:53:17,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-26 09:53:17,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-26 09:53:17,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-26 09:53:17,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-26 09:53:17,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-26 09:53:17,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-26 09:53:17,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-26 09:53:17,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-26 09:53:17,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-26 09:53:17,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-01-26 09:53:17,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-01-26 09:53:17,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-01-26 09:53:17,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-01-26 09:53:17,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-01-26 09:53:17,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-01-26 09:53:17,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-01-26 09:53:17,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-01-26 09:53:17,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-01-26 09:53:17,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-01-26 09:53:17,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-01-26 09:53:17,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-01-26 09:53:17,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-01-26 09:53:17,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-01-26 09:53:17,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-01-26 09:53:17,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-01-26 09:53:17,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-01-26 09:53:17,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-01-26 09:53:17,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-01-26 09:53:17,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-01-26 09:53:17,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-01-26 09:53:17,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-01-26 09:53:17,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-01-26 09:53:17,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-01-26 09:53:17,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-01-26 09:53:17,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-01-26 09:53:17,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-01-26 09:53:17,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-01-26 09:53:17,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-01-26 09:53:17,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-01-26 09:53:17,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-01-26 09:53:17,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-01-26 09:53:17,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-01-26 09:53:17,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-01-26 09:53:17,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-01-26 09:53:17,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-01-26 09:53:17,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-01-26 09:53:17,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-01-26 09:53:17,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-01-26 09:53:17,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-01-26 09:53:17,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-01-26 09:53:17,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-01-26 09:53:17,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-01-26 09:53:17,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-01-26 09:53:17,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-01-26 09:53:17,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-01-26 09:53:17,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-01-26 09:53:17,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-01-26 09:53:17,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-01-26 09:53:17,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-01-26 09:53:17,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-01-26 09:53:17,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-01-26 09:53:17,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-01-26 09:53:17,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-01-26 09:53:17,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-01-26 09:53:17,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-01-26 09:53:17,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-01-26 09:53:17,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-01-26 09:53:17,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-01-26 09:53:17,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-01-26 09:53:17,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-01-26 09:53:17,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-01-26 09:53:17,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-01-26 09:53:17,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-01-26 09:53:17,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-01-26 09:53:17,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-01-26 09:53:17,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-01-26 09:53:17,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-01-26 09:53:17,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-01-26 09:53:17,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-01-26 09:53:17,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-01-26 09:53:17,265 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=6.397985672269699E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014213381090374027); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-26 09:53:17,265 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1434310644:83 , srv1583863032:83 , srv1738071524:83 , srv1836144566:83 , srv2113622067:83 , srv224872179:83 , srv255989521:83 , srv809128489:83 , srv1013945090:84 , srv1031931493:84 , srv1046178159:84 , srv1048023265:84 , srv1055932790:84 , srv1075730593:84 , srv1079675115:84 , srv1085233182:84 , srv1113675187:84 , srv1164188193:84 , srv1183251659:84 , srv121490694:84 , srv1222082432:84 , srv1230193216:84 , srv1244903028:84 , srv1288211638:84 , srv1320833685:84 , srv1388957245:84 , srv1395247268:84 , srv142350950:84 , srv1431245026:84 , srv1440817235:84 , srv1476412779:84 , srv1579985826:84 , srv1597792747:84 , srv1609588180:84 , srv1610226970:84 , srv1620371410:84 , srv1641479215:84 , srv1661472461:84 , srv1673493308:84 , srv1700587406:84 , srv1714660213:84 , srv1744217215:84 , srv174932372:84 , srv1815656891:84 , srv1906295854:84 , srv1933359232:84 , srv1942392488:84 , srv1946505586:84 , srv1952553590:84 , srv1959463582:84 , srv1975876540:84 , srv19860228:84 , srv1991140319:84 , srv2008719378:84 , srv2016648032:84 , srv2019649024:84 , srv2042948209:84 , srv2047367625:84 , srv2047487867:84 , srv2067547083:84 , srv2095275402:84 , srv2104654889:84 , srv211380190:84 , srv219162643:84 , srv230105069:84 , srv277334020:84 , srv303507588:84 , srv311868537:84 , srv32830755:84 , srv380802452:84 , srv396424395:84 , srv429638735:84 , srv433109640:84 , srv44187094:84 , srv453207946:84 , srv497931549:84 , srv500967213:84 , srv538836256:84 , srv564575938:84 , srv637031711:84 , srv649573429:84 , srv694433629:84 , srv74685844:84 , srv801629165:84 , srv82828772:84 , srv874845007:84 , srv892856092:84 , srv928374692:84 , srv930233221:84 , srv930900634:84 , srv993973000:84 , srv1295200031:85 , srv1489240289:85 , srv1618606668:85 , srv1769119617:85 , srv2031938630:85 , srv411927662:85 , srv726211677:85 , srv809495613:85 , srv313511574:684 }
2024-01-26 09:53:17,305 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=286 (was 268) - SystemLoadAverage LEAK? -, ProcessCount=167 (was 167), AvailableMemoryMB=4824 (was 5763)
2024-01-26 09:53:17,314 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=286, ProcessCount=167, AvailableMemoryMB=4823
2024-01-26 09:53:17,315 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-01-26 09:53:17,315 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2024-01-26 09:53:17,331 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1026686423:84 , srv1137041053:84 , srv1148721617:84 , srv1267108429:84 , srv1307347687:84 , srv1320823177:84 , srv1463459235:84 , srv1473727159:84 , srv1536261305:84 , srv1623981737:84 , srv1757019257:84 , srv1798727713:84 , srv1817015601:84 , srv1824473351:84 , srv1826443867:84 , srv2005380373:84 , srv2005663433:84 , srv2117109610:84 , srv289399778:84 , srv303542828:84 , srv34800899:84 , srv43355220:84 , srv506990554:84 , srv722622408:84 , srv775694729:84 , srv894355349:84 , srv945461184:84 , srv968740264:84 , srv982010680:84 , srv2070861383:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2024-01-26 09:53:17,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-26 09:53:17,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1798727713=11, srv1536261305=8, srv1137041053=1, srv1473727159=7, srv722622408=24, srv1817015601=12, srv43355220=22, srv2005663433=16, srv2070861383=17, srv1757019257=10, srv775694729=25, srv303542828=20, srv1148721617=2, srv506990554=23, srv2005380373=15, srv1267108429=3, srv1824473351=13, srv34800899=21, srv1307347687=4, srv1463459235=6, srv289399778=19, srv968740264=28, srv894355349=26, srv1826443867=14, srv945461184=27, srv1320823177=5, srv1623981737=9, srv2117109610=18, srv982010680=29, srv1026686423=0} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-26 09:53:17,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-26 09:53:17,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-26 09:53:17,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-26 09:53:17,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-26 09:53:17,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-26 09:53:17,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-26 09:53:17,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-26 09:53:17,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-26 09:53:17,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-26 09:53:17,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-26 09:53:17,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-26 09:53:17,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-26 09:53:17,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-26 09:53:17,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-26 09:53:17,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-26 09:53:17,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-26 09:53:17,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-26 09:53:17,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-26 09:53:17,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-26 09:53:17,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-26 09:53:17,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-26 09:53:17,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-26 09:53:17,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-26 09:53:17,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-26 09:53:17,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-26 09:53:17,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-26 09:53:17,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-26 09:53:17,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-26 09:53:17,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-26 09:53:17,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-26 09:53:17,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-01-26 09:53:17,377 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1669049334189476, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-26 09:53:17,378 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912718955259534, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1669049334189476, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2024-01-26 09:55:17,357 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5700046 different iterations.  Found a solution that moves 1866 regions; Going from a computed imbalance of 0.9912718955259534 to a new imbalance of 4.3553370884861656E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6911111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-26 09:55:17,364 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1026686423:90 , srv1137041053:90 , srv1148721617:90 , srv1267108429:90 , srv1307347687:90 , srv1320823177:90 , srv1463459235:90 , srv1473727159:90 , srv1536261305:90 , srv1623981737:90 , srv1757019257:90 , srv1798727713:90 , srv1817015601:90 , srv1824473351:90 , srv1826443867:90 , srv2005380373:90 , srv2005663433:90 , srv2070861383:90 , srv2117109610:90 , srv289399778:90 , srv303542828:90 , srv34800899:90 , srv43355220:90 , srv506990554:90 , srv722622408:90 , srv775694729:90 , srv894355349:90 , srv945461184:90 , srv968740264:90 , srv982010680:90 }
2024-01-26 09:55:17,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-26 09:55:17,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1798727713=11, srv1536261305=8, srv1137041053=1, srv1473727159=7, srv722622408=24, srv1817015601=12, srv43355220=22, srv2005663433=16, srv2070861383=17, srv1757019257=10, srv775694729=25, srv303542828=20, srv1148721617=2, srv506990554=23, srv2005380373=15, srv1267108429=3, srv1824473351=13, srv34800899=21, srv1307347687=4, srv1463459235=6, srv289399778=19, srv968740264=28, srv894355349=26, srv1826443867=14, srv945461184=27, srv1320823177=5, srv1623981737=9, srv2117109610=18, srv982010680=29, srv1026686423=0} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-26 09:55:17,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-26 09:55:17,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-26 09:55:17,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-26 09:55:17,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-26 09:55:17,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-26 09:55:17,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-26 09:55:17,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-26 09:55:17,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-26 09:55:17,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-26 09:55:17,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-26 09:55:17,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-26 09:55:17,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-26 09:55:17,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-26 09:55:17,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-26 09:55:17,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-26 09:55:17,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-26 09:55:17,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-26 09:55:17,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-26 09:55:17,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-26 09:55:17,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-26 09:55:17,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-26 09:55:17,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-26 09:55:17,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-26 09:55:17,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-26 09:55:17,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-26 09:55:17,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-26 09:55:17,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-26 09:55:17,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-26 09:55:17,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-26 09:55:17,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-26 09:55:17,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-01-26 09:55:17,387 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-26 09:55:17,388 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1026686423:90 , srv1137041053:90 , srv1148721617:90 , srv1267108429:90 , srv1307347687:90 , srv1320823177:90 , srv1463459235:90 , srv1473727159:90 , srv1536261305:90 , srv1623981737:90 , srv1757019257:90 , srv1798727713:90 , srv1817015601:90 , srv1824473351:90 , srv1826443867:90 , srv2005380373:90 , srv2005663433:90 , srv2070861383:90 , srv2117109610:90 , srv289399778:90 , srv303542828:90 , srv34800899:90 , srv43355220:90 , srv506990554:90 , srv722622408:90 , srv775694729:90 , srv894355349:90 , srv945461184:90 , srv968740264:90 , srv982010680:90 }
2024-01-26 09:55:17,405 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=177 (was 286), ProcessCount=167 (was 167), AvailableMemoryMB=4229 (was 4823)