2023-12-10 06:53:14,970 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-12-10 06:53:15,302 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-12-10 06:53:15,420 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-12-10 06:53:15,489 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-12-10 06:53:15,496 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-12-10 06:53:15,510 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=260, ProcessCount=174, AvailableMemoryMB=7334
2023-12-10 06:53:15,512 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-12-10 06:53:15,512 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-12-10 06:53:15,624 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1002737072:84 , srv1059797347:84 , srv1074548841:84 , srv110374117:84 , srv1144602522:84 , srv1144706977:84 , srv1154660885:84 , srv115899613:84 , srv1159065765:84 , srv117112490:84 , srv1187944309:84 , srv1190651008:84 , srv1191054067:84 , srv1206810696:84 , srv1237613465:84 , srv1238859676:84 , srv1310880127:84 , srv1349058710:84 , srv1359779889:84 , srv1363603346:84 , srv1375715659:84 , srv1377754829:84 , srv1397900690:84 , srv1409070649:84 , srv1439143550:84 , srv147203917:84 , srv1472281930:84 , srv1483201740:84 , srv148520975:84 , srv151930983:84 , srv1526801961:84 , srv152712572:84 , srv1527379257:84 , srv1533886203:84 , srv1574427134:84 , srv1606611174:84 , srv1613756793:84 , srv1620913666:84 , srv1642145456:84 , srv1652046017:84 , srv1657331352:84 , srv1664692460:84 , srv1674515796:84 , srv1692634483:84 , srv1711758821:84 , srv1744605811:84 , srv178413793:84 , srv1789222972:84 , srv1804954364:84 , srv1839377213:84 , srv1855532105:84 , srv1908806478:84 , srv191933035:84 , srv1927408565:84 , srv1941880502:84 , srv1999476112:84 , srv2021324250:84 , srv2047456159:84 , srv2068308015:84 , srv2073181760:84 , srv2077176077:84 , srv2086397589:84 , srv226115136:84 , srv227049642:84 , srv227320157:84 , srv278177389:84 , srv311833441:84 , srv318018219:84 , srv389594382:84 , srv423546698:84 , srv454583765:84 , srv466407142:84 , srv470590190:84 , srv487806030:84 , srv530797426:84 , srv559578766:84 , srv580158734:84 , srv592001797:84 , srv629140647:84 , srv63090367:84 , srv656280358:84 , srv693572378:84 , srv738647871:84 , srv771788026:84 , srv796972645:84 , srv803868397:84 , srv821528062:84 , srv85197212:84 , srv857660666:84 , srv859185622:84 , srv866075580:84 , srv870617034:84 , srv886135650:84 , srv889806421:84 , srv90365891:84 , srv964977046:84 , srv965177910:84 , srv978262470:84 , srv981778157:84 , srv268866221:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-12-10 06:53:15,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-10 06:53:15,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv487806030=74, srv1349058710=17, srv1187944309=10, srv1059797347=1, srv1526801961=30, srv803868397=86, srv268866221=65, srv1002737072=0, srv1744605811=45, srv1674515796=42, srv1206810696=13, srv2086397589=61, srv1377754829=21, srv1855532105=50, srv117112490=9, srv470590190=73, srv796972645=85, srv859185622=90, srv1190651008=11, srv1310880127=16, srv423546698=70, srv191933035=52, srv821528062=87, srv693572378=82, srv1927408565=53, srv1839377213=49, srv592001797=78, srv1375715659=20, srv1804954364=48, srv1154660885=6, srv1642145456=38, srv85197212=88, srv580158734=77, srv278177389=66, srv1439143550=24, srv1533886203=33, srv1692634483=43, srv178413793=46, srv318018219=68, srv629140647=79, srv1999476112=55, srv886135650=93, srv738647871=83, srv147203917=25, srv559578766=76, srv1397900690=22, srv1620913666=37, srv2068308015=58, srv1606611174=35, srv1159065765=8, srv857660666=89, srv1527379257=32, srv866075580=91, srv1789222972=47, srv115899613=7, srv2021324250=56, srv311833441=67, srv148520975=28, srv1191054067=12, srv870617034=92, srv981778157=99, srv889806421=94, srv1363603346=19, srv454583765=71, srv1238859676=15, srv226115136=62, srv1144706977=5, srv2077176077=60, srv1711758821=44, srv530797426=75, srv656280358=81, srv1144602522=4, srv771788026=84, srv1613756793=36, srv110374117=3, srv1237613465=14, srv1652046017=39, srv1359779889=18, srv227320157=64, srv90365891=95, srv2073181760=59, srv227049642=63, srv1941880502=54, srv1664692460=41, srv2047456159=57, srv1074548841=2, srv466407142=72, srv151930983=29, srv1574427134=34, srv389594382=69, srv965177910=97, srv1409070649=23, srv1472281930=26, srv1483201740=27, srv1908806478=51, srv63090367=80, srv1657331352=40, srv152712572=31, srv964977046=96, srv978262470=98} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-10 06:53:15,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-10 06:53:15,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-10 06:53:15,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-10 06:53:15,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-10 06:53:15,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-10 06:53:15,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-10 06:53:15,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-10 06:53:15,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-10 06:53:15,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-10 06:53:15,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-10 06:53:15,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-10 06:53:15,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-10 06:53:15,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-10 06:53:15,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-10 06:53:15,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-10 06:53:15,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-10 06:53:15,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-10 06:53:15,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-10 06:53:15,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-10 06:53:15,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-10 06:53:15,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-10 06:53:15,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-10 06:53:15,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-10 06:53:15,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-10 06:53:15,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-10 06:53:15,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-10 06:53:15,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-10 06:53:15,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-10 06:53:15,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-10 06:53:15,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-10 06:53:15,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-10 06:53:15,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-10 06:53:15,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-10 06:53:15,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-10 06:53:15,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-10 06:53:15,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-10 06:53:15,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-10 06:53:15,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-10 06:53:15,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-10 06:53:15,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-10 06:53:15,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-10 06:53:15,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-10 06:53:15,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-10 06:53:15,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-10 06:53:15,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-10 06:53:15,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-10 06:53:15,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-10 06:53:15,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-10 06:53:15,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-10 06:53:15,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-10 06:53:15,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-10 06:53:15,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-10 06:53:15,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-10 06:53:15,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-10 06:53:15,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-10 06:53:15,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-10 06:53:15,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-10 06:53:15,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-10 06:53:15,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-10 06:53:15,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-10 06:53:15,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-10 06:53:15,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-10 06:53:15,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-10 06:53:15,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-10 06:53:15,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-10 06:53:15,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-10 06:53:15,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-10 06:53:15,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-10 06:53:15,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-10 06:53:15,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-10 06:53:15,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-10 06:53:15,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-10 06:53:15,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-10 06:53:15,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-10 06:53:15,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-10 06:53:15,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-10 06:53:15,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-10 06:53:15,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-10 06:53:15,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-10 06:53:15,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-10 06:53:15,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-10 06:53:15,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-10 06:53:15,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-10 06:53:15,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-10 06:53:15,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-10 06:53:15,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-10 06:53:15,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-10 06:53:15,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-10 06:53:15,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-10 06:53:15,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-10 06:53:15,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-10 06:53:15,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-10 06:53:15,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-10 06:53:15,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-10 06:53:15,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-10 06:53:15,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-10 06:53:15,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-10 06:53:15,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-10 06:53:15,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-10 06:53:15,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-10 06:53:15,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-10 06:53:15,965 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-10 06:53:15,966 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-12-10 06:53:15,966 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-12-10 06:53:16,366 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 591 ms to try 5000 different iterations.  Found a solution that moves 4727 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17501315778039878. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0394715055526432, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05361876482761056, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5252222222222223, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0394715055526432, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17033333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2355, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-10 06:53:16,382 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv63090367:82 , srv1159065765:83 , srv1238859676:83 , srv1855532105:83 , srv2021324250:83 , srv226115136:83 , srv487806030:83 , srv1002737072:84 , srv1409070649:84 , srv1472281930:84 , srv1674515796:84 , srv1692634483:84 , srv2068308015:84 , srv2073181760:84 , srv530797426:84 , srv771788026:84 , srv870617034:84 , srv889806421:84 , srv110374117:85 , srv1144706977:85 , srv115899613:85 , srv1375715659:85 , srv1377754829:85 , srv1397900690:85 , srv1483201740:85 , srv148520975:85 , srv152712572:85 , srv1613756793:85 , srv1744605811:85 , srv2077176077:85 , srv2086397589:85 , srv466407142:85 , srv656280358:85 , srv693572378:85 , srv803868397:85 , srv90365891:85 , srv964977046:85 , srv981778157:85 , srv1144602522:86 , srv1154660885:86 , srv1237613465:86 , srv1359779889:86 , srv1363603346:86 , srv1526801961:86 , srv1533886203:86 , srv1574427134:86 , srv1620913666:86 , srv1652046017:86 , srv1664692460:86 , srv1804954364:86 , srv191933035:86 , srv2047456159:86 , srv470590190:86 , srv580158734:86 , srv592001797:86 , srv821528062:86 , srv857660666:86 , srv1310880127:87 , srv1439143550:87 , srv1642145456:87 , srv278177389:87 , srv311833441:87 , srv318018219:87 , srv389594382:87 , srv423546698:87 , srv629140647:87 , srv886135650:87 , srv978262470:87 , srv1187944309:88 , srv1206810696:88 , srv147203917:88 , srv1527379257:88 , srv1711758821:88 , srv1908806478:88 , srv1927408565:88 , srv1999476112:88 , srv227049642:88 , srv454583765:88 , srv559578766:88 , srv117112490:89 , srv1190651008:89 , srv1349058710:89 , srv1606611174:89 , srv1657331352:89 , srv178413793:89 , srv1789222972:89 , srv227320157:89 , srv738647871:89 , srv859185622:89 , srv866075580:89 , srv965177910:89 , srv1059797347:90 , srv1191054067:90 , srv1839377213:90 , srv85197212:90 , srv1074548841:91 , srv796972645:91 , srv1941880502:92 , srv151930983:94 , srv268866221:441 }
2023-12-10 06:53:16,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-10 06:53:16,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv487806030=74, srv1349058710=17, srv1187944309=10, srv1059797347=1, srv1526801961=30, srv803868397=86, srv268866221=65, srv1002737072=0, srv1744605811=45, srv1674515796=42, srv1206810696=13, srv2086397589=61, srv1377754829=21, srv1855532105=50, srv117112490=9, srv470590190=73, srv796972645=85, srv859185622=90, srv1190651008=11, srv1310880127=16, srv423546698=70, srv191933035=52, srv821528062=87, srv693572378=82, srv1927408565=53, srv1839377213=49, srv592001797=78, srv1375715659=20, srv1804954364=48, srv1154660885=6, srv1642145456=38, srv85197212=88, srv580158734=77, srv278177389=66, srv1439143550=24, srv1533886203=33, srv1692634483=43, srv178413793=46, srv318018219=68, srv629140647=79, srv1999476112=55, srv886135650=93, srv738647871=83, srv147203917=25, srv559578766=76, srv1397900690=22, srv1620913666=37, srv2068308015=58, srv1606611174=35, srv1159065765=8, srv857660666=89, srv1527379257=32, srv866075580=91, srv1789222972=47, srv115899613=7, srv2021324250=56, srv311833441=67, srv148520975=28, srv1191054067=12, srv870617034=92, srv981778157=99, srv889806421=94, srv1363603346=19, srv454583765=71, srv1238859676=15, srv226115136=62, srv1144706977=5, srv2077176077=60, srv1711758821=44, srv530797426=75, srv656280358=81, srv1144602522=4, srv771788026=84, srv1613756793=36, srv110374117=3, srv1237613465=14, srv1652046017=39, srv1359779889=18, srv227320157=64, srv90365891=95, srv2073181760=59, srv227049642=63, srv1941880502=54, srv1664692460=41, srv2047456159=57, srv1074548841=2, srv466407142=72, srv151930983=29, srv1574427134=34, srv389594382=69, srv965177910=97, srv1409070649=23, srv1472281930=26, srv1483201740=27, srv1908806478=51, srv63090367=80, srv1657331352=40, srv152712572=31, srv964977046=96, srv978262470=98} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-10 06:53:16,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-10 06:53:16,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-10 06:53:16,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-10 06:53:16,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-10 06:53:16,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-10 06:53:16,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-10 06:53:16,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-10 06:53:16,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-10 06:53:16,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-10 06:53:16,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-10 06:53:16,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-10 06:53:16,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-10 06:53:16,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-10 06:53:16,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-10 06:53:16,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-10 06:53:16,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-10 06:53:16,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-10 06:53:16,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-10 06:53:16,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-10 06:53:16,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-10 06:53:16,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-10 06:53:16,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-10 06:53:16,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-10 06:53:16,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-10 06:53:16,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-10 06:53:16,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-10 06:53:16,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-10 06:53:16,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-10 06:53:16,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-10 06:53:16,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-10 06:53:16,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-10 06:53:16,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-10 06:53:16,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-10 06:53:16,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-10 06:53:16,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-10 06:53:16,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-10 06:53:16,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-10 06:53:16,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-10 06:53:16,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-10 06:53:16,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-10 06:53:16,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-10 06:53:16,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-10 06:53:16,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-10 06:53:16,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-10 06:53:16,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-10 06:53:16,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-10 06:53:16,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-10 06:53:16,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-10 06:53:16,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-10 06:53:16,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-10 06:53:16,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-10 06:53:16,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-10 06:53:16,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-10 06:53:16,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-10 06:53:16,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-10 06:53:16,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-10 06:53:16,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-10 06:53:16,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-10 06:53:16,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-10 06:53:16,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-10 06:53:16,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-10 06:53:16,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-10 06:53:16,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-10 06:53:16,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-10 06:53:16,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-10 06:53:16,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-10 06:53:16,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-10 06:53:16,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-10 06:53:16,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-10 06:53:16,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-10 06:53:16,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-10 06:53:16,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-10 06:53:16,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-10 06:53:16,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-10 06:53:16,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-10 06:53:16,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-10 06:53:16,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-10 06:53:16,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-10 06:53:16,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-10 06:53:16,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-10 06:53:16,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-10 06:53:16,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-10 06:53:16,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-10 06:53:16,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-10 06:53:16,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-10 06:53:16,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-10 06:53:16,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-10 06:53:16,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-10 06:53:16,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-10 06:53:16,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-10 06:53:16,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-10 06:53:16,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-10 06:53:16,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-10 06:53:16,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-10 06:53:16,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-10 06:53:16,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-10 06:53:16,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-10 06:53:16,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-10 06:53:16,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-10 06:53:16,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-10 06:53:16,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-10 06:53:16,529 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0394715055526432, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05361876482761056, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0394715055526432, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17033333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2355, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-10 06:53:16,530 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-12-10 06:53:16,530 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17498005861895624, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0394715055526432, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05361876482761056, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0394715055526432, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17033333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2355, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-12-10 06:53:16,826 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 397 ms to try 5000 different iterations.  Found a solution that moves 2569 regions; Going from a computed imbalance of 0.17498005861895624 to a new imbalance of 0.01412990497539555. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020716074799094566); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02951915431343641, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.28544444444444445, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020716074799094566); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010833333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04583333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-10 06:53:16,838 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1191054067:79 , srv1941880502:80 , srv796972645:80 , srv1074548841:81 , srv151930983:81 , srv859185622:81 , srv1059797347:82 , srv1310880127:82 , srv1657331352:82 , srv178413793:82 , srv227320157:82 , srv117112490:83 , srv1187944309:83 , srv1190651008:83 , srv1206810696:83 , srv1999476112:83 , srv318018219:83 , srv559578766:83 , srv85197212:83 , srv866075580:83 , srv886135650:83 , srv965177910:83 , srv1642145456:84 , srv1711758821:84 , srv311833441:84 , srv592001797:84 , srv738647871:84 , srv1349058710:85 , srv1359779889:85 , srv1439143550:85 , srv1526801961:85 , srv1613756793:85 , srv1652046017:85 , srv1789222972:85 , srv191933035:85 , srv1927408565:85 , srv2077176077:85 , srv227049642:85 , srv454583765:85 , srv629140647:85 , srv693572378:85 , srv821528062:85 , srv857660666:85 , srv1154660885:86 , srv115899613:86 , srv1237613465:86 , srv1377754829:86 , srv1397900690:86 , srv147203917:86 , srv1483201740:86 , srv1606611174:86 , srv1620913666:86 , srv1664692460:86 , srv1744605811:86 , srv1839377213:86 , srv1908806478:86 , srv278177389:86 , srv466407142:86 , srv656280358:86 , srv978262470:86 , srv1002737072:87 , srv110374117:87 , srv1144602522:87 , srv1409070649:87 , srv148520975:87 , srv152712572:87 , srv1574427134:87 , srv1674515796:87 , srv2047456159:87 , srv2086397589:87 , srv470590190:87 , srv530797426:87 , srv580158734:87 , srv771788026:87 , srv870617034:87 , srv1144706977:88 , srv1159065765:88 , srv1375715659:88 , srv1527379257:88 , srv1533886203:88 , srv1692634483:88 , srv1804954364:88 , srv1855532105:88 , srv2068308015:88 , srv389594382:88 , srv423546698:88 , srv90365891:88 , srv981778157:88 , srv1363603346:89 , srv2073181760:89 , srv487806030:89 , srv803868397:89 , srv889806421:89 , srv964977046:89 , srv1238859676:90 , srv1472281930:90 , srv226115136:90 , srv2021324250:91 , srv63090367:92 , srv268866221:517 }
2023-12-10 06:53:16,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-10 06:53:16,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv487806030=74, srv1349058710=17, srv1187944309=10, srv1059797347=1, srv1526801961=30, srv803868397=86, srv268866221=65, srv1002737072=0, srv1744605811=45, srv1674515796=42, srv1206810696=13, srv2086397589=61, srv1377754829=21, srv1855532105=50, srv117112490=9, srv470590190=73, srv796972645=85, srv859185622=90, srv1190651008=11, srv1310880127=16, srv423546698=70, srv191933035=52, srv821528062=87, srv693572378=82, srv1927408565=53, srv1839377213=49, srv592001797=78, srv1375715659=20, srv1804954364=48, srv1154660885=6, srv1642145456=38, srv85197212=88, srv580158734=77, srv278177389=66, srv1439143550=24, srv1533886203=33, srv1692634483=43, srv178413793=46, srv318018219=68, srv629140647=79, srv1999476112=55, srv886135650=93, srv738647871=83, srv147203917=25, srv559578766=76, srv1397900690=22, srv1620913666=37, srv2068308015=58, srv1606611174=35, srv1159065765=8, srv857660666=89, srv1527379257=32, srv866075580=91, srv1789222972=47, srv115899613=7, srv2021324250=56, srv311833441=67, srv148520975=28, srv1191054067=12, srv870617034=92, srv981778157=99, srv889806421=94, srv1363603346=19, srv454583765=71, srv1238859676=15, srv226115136=62, srv1144706977=5, srv2077176077=60, srv1711758821=44, srv530797426=75, srv656280358=81, srv1144602522=4, srv771788026=84, srv1613756793=36, srv110374117=3, srv1237613465=14, srv1652046017=39, srv1359779889=18, srv227320157=64, srv90365891=95, srv2073181760=59, srv227049642=63, srv1941880502=54, srv1664692460=41, srv2047456159=57, srv1074548841=2, srv466407142=72, srv151930983=29, srv1574427134=34, srv389594382=69, srv965177910=97, srv1409070649=23, srv1472281930=26, srv1483201740=27, srv1908806478=51, srv63090367=80, srv1657331352=40, srv152712572=31, srv964977046=96, srv978262470=98} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-10 06:53:16,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-10 06:53:16,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-10 06:53:16,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-10 06:53:16,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-10 06:53:16,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-10 06:53:16,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-10 06:53:16,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-10 06:53:16,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-10 06:53:16,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-10 06:53:16,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-10 06:53:16,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-10 06:53:16,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-10 06:53:16,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-10 06:53:16,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-10 06:53:16,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-10 06:53:16,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-10 06:53:16,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-10 06:53:16,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-10 06:53:16,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-10 06:53:16,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-10 06:53:16,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-10 06:53:16,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-10 06:53:16,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-10 06:53:16,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-10 06:53:16,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-10 06:53:16,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-10 06:53:16,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-10 06:53:16,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-10 06:53:16,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-10 06:53:16,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-10 06:53:16,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-10 06:53:16,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-10 06:53:16,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-10 06:53:16,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-10 06:53:16,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-10 06:53:16,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-10 06:53:16,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-10 06:53:16,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-10 06:53:16,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-10 06:53:16,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-10 06:53:16,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-10 06:53:16,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-10 06:53:16,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-10 06:53:16,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-10 06:53:16,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-10 06:53:16,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-10 06:53:16,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-10 06:53:16,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-10 06:53:16,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-10 06:53:16,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-10 06:53:16,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-10 06:53:16,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-10 06:53:16,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-10 06:53:16,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-10 06:53:16,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-10 06:53:16,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-10 06:53:16,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-10 06:53:16,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-10 06:53:16,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-10 06:53:16,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-10 06:53:16,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-10 06:53:16,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-10 06:53:16,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-10 06:53:16,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-10 06:53:16,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-10 06:53:16,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-10 06:53:16,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-10 06:53:16,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-10 06:53:16,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-10 06:53:16,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-10 06:53:16,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-10 06:53:16,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-10 06:53:16,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-10 06:53:16,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-10 06:53:16,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-10 06:53:16,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-10 06:53:16,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-10 06:53:16,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-10 06:53:16,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-10 06:53:16,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-10 06:53:16,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-10 06:53:16,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-10 06:53:16,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-10 06:53:16,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-10 06:53:16,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-10 06:53:16,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-10 06:53:16,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-10 06:53:16,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-10 06:53:16,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-10 06:53:16,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-10 06:53:16,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-10 06:53:16,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-10 06:53:16,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-10 06:53:16,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-10 06:53:16,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-10 06:53:16,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-10 06:53:16,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-10 06:53:16,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-10 06:53:16,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-10 06:53:16,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-10 06:53:16,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-10 06:53:17,039 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020716074799094566); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02951915431343641, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020716074799094566); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010833333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04583333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-10 06:53:17,039 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-12-10 06:53:17,040 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.014111916452919149, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020716074799094566); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02951915431343641, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020716074799094566); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010833333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04583333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-12-10 06:53:17,452 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 531 ms to try 5000 different iterations.  Found a solution that moves 950 regions; Going from a computed imbalance of 0.014111916452919149 to a new imbalance of 6.55736039319893E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007560703336463659); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00943998706461097); MoveCostFunction : (multiplier=7.0, imbalance=0.10555555555555556, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007560703336463659); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-10 06:53:17,457 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1527379257:81 , srv389594382:81 , srv151930983:82 , srv1606611174:82 , srv63090367:82 , srv1363603346:83 , srv147203917:83 , srv1472281930:83 , srv1533886203:83 , srv1789222972:83 , srv1839377213:83 , srv2021324250:83 , srv2073181760:83 , srv423546698:83 , srv1144706977:84 , srv1349058710:84 , srv148520975:84 , srv1804954364:84 , srv1908806478:84 , srv1927408565:84 , srv2047456159:84 , srv278177389:84 , srv454583765:84 , srv580158734:84 , srv738647871:84 , srv803868397:84 , srv85197212:84 , srv866075580:84 , srv90365891:84 , srv964977046:84 , srv1074548841:85 , srv1144602522:85 , srv115899613:85 , srv1159065765:85 , srv117112490:85 , srv1190651008:85 , srv1238859676:85 , srv1375715659:85 , srv152712572:85 , srv1574427134:85 , srv1664692460:85 , srv1692634483:85 , srv1711758821:85 , srv1941880502:85 , srv2068308015:85 , srv2086397589:85 , srv226115136:85 , srv227049642:85 , srv487806030:85 , srv629140647:85 , srv857660666:85 , srv889806421:85 , srv965177910:85 , srv978262470:85 , srv981778157:85 , srv1002737072:86 , srv1059797347:86 , srv110374117:86 , srv1154660885:86 , srv1187944309:86 , srv1206810696:86 , srv1237613465:86 , srv1359779889:86 , srv1397900690:86 , srv1439143550:86 , srv1483201740:86 , srv1620913666:86 , srv1642145456:86 , srv1652046017:86 , srv1744605811:86 , srv178413793:86 , srv191933035:86 , srv1999476112:86 , srv227320157:86 , srv311833441:86 , srv318018219:86 , srv470590190:86 , srv530797426:86 , srv656280358:86 , srv796972645:86 , srv1191054067:87 , srv1526801961:87 , srv1613756793:87 , srv1657331352:87 , srv1674515796:87 , srv1855532105:87 , srv466407142:87 , srv559578766:87 , srv693572378:87 , srv771788026:87 , srv821528062:87 , srv859185622:87 , srv870617034:87 , srv1310880127:88 , srv1377754829:88 , srv1409070649:88 , srv2077176077:88 , srv592001797:88 , srv886135650:88 , srv268866221:567 }
2023-12-10 06:53:17,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-10 06:53:17,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv487806030=74, srv1349058710=17, srv1187944309=10, srv1059797347=1, srv1526801961=30, srv803868397=86, srv268866221=65, srv1002737072=0, srv1744605811=45, srv1674515796=42, srv1206810696=13, srv2086397589=61, srv1377754829=21, srv1855532105=50, srv117112490=9, srv470590190=73, srv796972645=85, srv859185622=90, srv1190651008=11, srv1310880127=16, srv423546698=70, srv191933035=52, srv821528062=87, srv693572378=82, srv1927408565=53, srv1839377213=49, srv592001797=78, srv1375715659=20, srv1804954364=48, srv1154660885=6, srv1642145456=38, srv85197212=88, srv580158734=77, srv278177389=66, srv1439143550=24, srv1533886203=33, srv1692634483=43, srv178413793=46, srv318018219=68, srv629140647=79, srv1999476112=55, srv886135650=93, srv738647871=83, srv147203917=25, srv559578766=76, srv1397900690=22, srv1620913666=37, srv2068308015=58, srv1606611174=35, srv1159065765=8, srv857660666=89, srv1527379257=32, srv866075580=91, srv1789222972=47, srv115899613=7, srv2021324250=56, srv311833441=67, srv148520975=28, srv1191054067=12, srv870617034=92, srv981778157=99, srv889806421=94, srv1363603346=19, srv454583765=71, srv1238859676=15, srv226115136=62, srv1144706977=5, srv2077176077=60, srv1711758821=44, srv530797426=75, srv656280358=81, srv1144602522=4, srv771788026=84, srv1613756793=36, srv110374117=3, srv1237613465=14, srv1652046017=39, srv1359779889=18, srv227320157=64, srv90365891=95, srv2073181760=59, srv227049642=63, srv1941880502=54, srv1664692460=41, srv2047456159=57, srv1074548841=2, srv466407142=72, srv151930983=29, srv1574427134=34, srv389594382=69, srv965177910=97, srv1409070649=23, srv1472281930=26, srv1483201740=27, srv1908806478=51, srv63090367=80, srv1657331352=40, srv152712572=31, srv964977046=96, srv978262470=98} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-10 06:53:17,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-10 06:53:17,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-10 06:53:17,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-10 06:53:17,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-10 06:53:17,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-10 06:53:17,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-10 06:53:17,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-10 06:53:17,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-10 06:53:17,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-10 06:53:17,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-10 06:53:17,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-10 06:53:17,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-10 06:53:17,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-10 06:53:17,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-10 06:53:17,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-10 06:53:17,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-10 06:53:17,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-10 06:53:17,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-10 06:53:17,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-10 06:53:17,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-10 06:53:17,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-10 06:53:17,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-10 06:53:17,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-10 06:53:17,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-10 06:53:17,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-10 06:53:17,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-10 06:53:17,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-10 06:53:17,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-10 06:53:17,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-10 06:53:17,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-10 06:53:17,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-10 06:53:17,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-10 06:53:17,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-10 06:53:17,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-10 06:53:17,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-10 06:53:17,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-10 06:53:17,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-10 06:53:17,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-10 06:53:17,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-10 06:53:17,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-10 06:53:17,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-10 06:53:17,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-10 06:53:17,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-10 06:53:17,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-10 06:53:17,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-10 06:53:17,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-10 06:53:17,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-10 06:53:17,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-10 06:53:17,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-10 06:53:17,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-10 06:53:17,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-10 06:53:17,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-10 06:53:17,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-10 06:53:17,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-10 06:53:17,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-10 06:53:17,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-10 06:53:17,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-10 06:53:17,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-10 06:53:17,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-10 06:53:17,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-10 06:53:17,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-10 06:53:17,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-10 06:53:17,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-10 06:53:17,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-10 06:53:17,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-10 06:53:17,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-10 06:53:17,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-10 06:53:17,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-10 06:53:17,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-10 06:53:17,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-10 06:53:17,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-10 06:53:17,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-10 06:53:17,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-10 06:53:17,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-10 06:53:17,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-10 06:53:17,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-10 06:53:17,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-10 06:53:17,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-10 06:53:17,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-10 06:53:17,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-10 06:53:17,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-10 06:53:17,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-10 06:53:17,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-10 06:53:17,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-10 06:53:17,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-10 06:53:17,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-10 06:53:17,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-10 06:53:17,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-10 06:53:17,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-10 06:53:17,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-10 06:53:17,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-10 06:53:17,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-10 06:53:17,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-10 06:53:17,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-10 06:53:17,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-10 06:53:17,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-10 06:53:17,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-10 06:53:17,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-10 06:53:17,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-10 06:53:17,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-10 06:53:17,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-10 06:53:17,589 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007560703336463659); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00943998706461097); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007560703336463659); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-10 06:53:17,589 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-12-10 06:53:17,589 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=6.490839971429448E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007560703336463659); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00943998706461097); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007560703336463659); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-12-10 06:53:17,908 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 414 ms to try 5000 different iterations.  Found a solution that moves 239 regions; Going from a computed imbalance of 6.490839971429448E-4 to a new imbalance of 1.2479193175736975E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.178341294946416E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0019534423844256277); MoveCostFunction : (multiplier=7.0, imbalance=0.026555555555555554, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.178341294946416E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-10 06:53:17,909 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1377754829:83 , srv1409070649:83 , srv1839377213:83 , srv470590190:83 , srv1059797347:84 , srv110374117:84 , srv1144602522:84 , srv1154660885:84 , srv1187944309:84 , srv1206810696:84 , srv1237613465:84 , srv1238859676:84 , srv1363603346:84 , srv1375715659:84 , srv1439143550:84 , srv1574427134:84 , srv1606611174:84 , srv1620913666:84 , srv1657331352:84 , srv1674515796:84 , srv1804954364:84 , srv1855532105:84 , srv1908806478:84 , srv2077176077:84 , srv226115136:84 , srv227049642:84 , srv311833441:84 , srv423546698:84 , srv466407142:84 , srv559578766:84 , srv592001797:84 , srv771788026:84 , srv796972645:84 , srv803868397:84 , srv821528062:84 , srv886135650:84 , srv889806421:84 , srv978262470:84 , srv981778157:84 , srv1002737072:85 , srv1074548841:85 , srv1144706977:85 , srv117112490:85 , srv1190651008:85 , srv1310880127:85 , srv1349058710:85 , srv1359779889:85 , srv1397900690:85 , srv147203917:85 , srv1472281930:85 , srv1483201740:85 , srv151930983:85 , srv1526801961:85 , srv152712572:85 , srv1527379257:85 , srv1533886203:85 , srv1613756793:85 , srv1642145456:85 , srv1652046017:85 , srv1664692460:85 , srv1692634483:85 , srv1711758821:85 , srv1744605811:85 , srv178413793:85 , srv1789222972:85 , srv191933035:85 , srv1927408565:85 , srv1941880502:85 , srv1999476112:85 , srv2047456159:85 , srv2086397589:85 , srv278177389:85 , srv389594382:85 , srv454583765:85 , srv487806030:85 , srv530797426:85 , srv580158734:85 , srv629140647:85 , srv656280358:85 , srv693572378:85 , srv738647871:85 , srv85197212:85 , srv857660666:85 , srv859185622:85 , srv870617034:85 , srv90365891:85 , srv964977046:85 , srv965177910:85 , srv115899613:86 , srv1159065765:86 , srv1191054067:86 , srv148520975:86 , srv2021324250:86 , srv2068308015:86 , srv2073181760:86 , srv227320157:86 , srv318018219:86 , srv63090367:86 , srv866075580:86 , srv268866221:617 }
2023-12-10 06:53:17,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-10 06:53:17,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv487806030=74, srv1349058710=17, srv1187944309=10, srv1059797347=1, srv1526801961=30, srv803868397=86, srv268866221=65, srv1002737072=0, srv1744605811=45, srv1674515796=42, srv1206810696=13, srv2086397589=61, srv1377754829=21, srv1855532105=50, srv117112490=9, srv470590190=73, srv796972645=85, srv859185622=90, srv1190651008=11, srv1310880127=16, srv423546698=70, srv191933035=52, srv821528062=87, srv693572378=82, srv1927408565=53, srv1839377213=49, srv592001797=78, srv1375715659=20, srv1804954364=48, srv1154660885=6, srv1642145456=38, srv85197212=88, srv580158734=77, srv278177389=66, srv1439143550=24, srv1533886203=33, srv1692634483=43, srv178413793=46, srv318018219=68, srv629140647=79, srv1999476112=55, srv886135650=93, srv738647871=83, srv147203917=25, srv559578766=76, srv1397900690=22, srv1620913666=37, srv2068308015=58, srv1606611174=35, srv1159065765=8, srv857660666=89, srv1527379257=32, srv866075580=91, srv1789222972=47, srv115899613=7, srv2021324250=56, srv311833441=67, srv148520975=28, srv1191054067=12, srv870617034=92, srv981778157=99, srv889806421=94, srv1363603346=19, srv454583765=71, srv1238859676=15, srv226115136=62, srv1144706977=5, srv2077176077=60, srv1711758821=44, srv530797426=75, srv656280358=81, srv1144602522=4, srv771788026=84, srv1613756793=36, srv110374117=3, srv1237613465=14, srv1652046017=39, srv1359779889=18, srv227320157=64, srv90365891=95, srv2073181760=59, srv227049642=63, srv1941880502=54, srv1664692460=41, srv2047456159=57, srv1074548841=2, srv466407142=72, srv151930983=29, srv1574427134=34, srv389594382=69, srv965177910=97, srv1409070649=23, srv1472281930=26, srv1483201740=27, srv1908806478=51, srv63090367=80, srv1657331352=40, srv152712572=31, srv964977046=96, srv978262470=98} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-10 06:53:17,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-10 06:53:17,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-10 06:53:17,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-10 06:53:17,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-10 06:53:17,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-10 06:53:17,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-10 06:53:17,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-10 06:53:17,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-10 06:53:17,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-10 06:53:17,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-10 06:53:17,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-10 06:53:17,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-10 06:53:17,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-10 06:53:17,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-10 06:53:17,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-10 06:53:17,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-10 06:53:17,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-10 06:53:17,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-10 06:53:17,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-10 06:53:17,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-10 06:53:17,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-10 06:53:17,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-10 06:53:17,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-10 06:53:17,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-10 06:53:17,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-10 06:53:17,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-10 06:53:17,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-10 06:53:17,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-10 06:53:17,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-10 06:53:17,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-10 06:53:17,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-10 06:53:17,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-10 06:53:17,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-10 06:53:17,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-10 06:53:17,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-10 06:53:17,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-10 06:53:17,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-10 06:53:17,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-10 06:53:17,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-10 06:53:17,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-10 06:53:17,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-10 06:53:17,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-10 06:53:17,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-10 06:53:17,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-10 06:53:17,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-10 06:53:17,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-10 06:53:17,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-10 06:53:17,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-10 06:53:17,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-10 06:53:17,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-10 06:53:17,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-10 06:53:17,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-10 06:53:17,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-10 06:53:17,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-10 06:53:17,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-10 06:53:17,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-10 06:53:17,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-10 06:53:17,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-10 06:53:17,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-10 06:53:17,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-10 06:53:17,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-10 06:53:17,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-10 06:53:17,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-10 06:53:17,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-10 06:53:17,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-10 06:53:17,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-10 06:53:17,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-10 06:53:17,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-10 06:53:17,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-10 06:53:17,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-10 06:53:17,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-10 06:53:17,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-10 06:53:17,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-10 06:53:17,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-10 06:53:17,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-10 06:53:17,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-10 06:53:17,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-10 06:53:17,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-10 06:53:17,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-10 06:53:17,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-10 06:53:17,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-10 06:53:17,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-10 06:53:17,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-10 06:53:17,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-10 06:53:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-10 06:53:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-10 06:53:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-10 06:53:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-10 06:53:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-10 06:53:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-10 06:53:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-10 06:53:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-10 06:53:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-10 06:53:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-10 06:53:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-10 06:53:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-10 06:53:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-10 06:53:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-10 06:53:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-10 06:53:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-10 06:53:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-10 06:53:18,067 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.0805679407009976E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.178341294946416E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0019534423844256277); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.178341294946416E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-10 06:53:18,067 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1377754829:83 , srv1409070649:83 , srv1839377213:83 , srv470590190:83 , srv1059797347:84 , srv110374117:84 , srv1144602522:84 , srv1154660885:84 , srv1187944309:84 , srv1206810696:84 , srv1237613465:84 , srv1238859676:84 , srv1363603346:84 , srv1375715659:84 , srv1439143550:84 , srv1574427134:84 , srv1606611174:84 , srv1620913666:84 , srv1657331352:84 , srv1674515796:84 , srv1804954364:84 , srv1855532105:84 , srv1908806478:84 , srv2077176077:84 , srv226115136:84 , srv227049642:84 , srv311833441:84 , srv423546698:84 , srv466407142:84 , srv559578766:84 , srv592001797:84 , srv771788026:84 , srv796972645:84 , srv803868397:84 , srv821528062:84 , srv886135650:84 , srv889806421:84 , srv978262470:84 , srv981778157:84 , srv1002737072:85 , srv1074548841:85 , srv1144706977:85 , srv117112490:85 , srv1190651008:85 , srv1310880127:85 , srv1349058710:85 , srv1359779889:85 , srv1397900690:85 , srv147203917:85 , srv1472281930:85 , srv1483201740:85 , srv151930983:85 , srv1526801961:85 , srv152712572:85 , srv1527379257:85 , srv1533886203:85 , srv1613756793:85 , srv1642145456:85 , srv1652046017:85 , srv1664692460:85 , srv1692634483:85 , srv1711758821:85 , srv1744605811:85 , srv178413793:85 , srv1789222972:85 , srv191933035:85 , srv1927408565:85 , srv1941880502:85 , srv1999476112:85 , srv2047456159:85 , srv2086397589:85 , srv278177389:85 , srv389594382:85 , srv454583765:85 , srv487806030:85 , srv530797426:85 , srv580158734:85 , srv629140647:85 , srv656280358:85 , srv693572378:85 , srv738647871:85 , srv85197212:85 , srv857660666:85 , srv859185622:85 , srv870617034:85 , srv90365891:85 , srv964977046:85 , srv965177910:85 , srv115899613:86 , srv1159065765:86 , srv1191054067:86 , srv148520975:86 , srv2021324250:86 , srv2068308015:86 , srv2073181760:86 , srv227320157:86 , srv318018219:86 , srv63090367:86 , srv866075580:86 , srv268866221:617 }
2023-12-10 06:53:18,097 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=311 (was 260) - SystemLoadAverage LEAK? -, ProcessCount=174 (was 174), AvailableMemoryMB=6513 (was 7334)
2023-12-10 06:53:18,105 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=311, ProcessCount=174, AvailableMemoryMB=6513
2023-12-10 06:53:18,105 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-12-10 06:53:18,106 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-12-10 06:53:18,114 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv110108989:84 , srv1114678074:84 , srv1127814338:84 , srv1173518582:84 , srv1216672720:84 , srv1289766122:84 , srv1425293750:84 , srv1526335495:84 , srv1708278962:84 , srv172236876:84 , srv1727515349:84 , srv1728258428:84 , srv1801902772:84 , srv190423615:84 , srv192126125:84 , srv1936576159:84 , srv196930741:84 , srv20786253:84 , srv2117922725:84 , srv233928461:84 , srv312855228:84 , srv349534445:84 , srv40377272:84 , srv470084322:84 , srv545343646:84 , srv607178195:84 , srv727914507:84 , srv846353458:84 , srv886417441:84 , srv1398041824:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-12-10 06:53:18,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-10 06:53:18,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv40377272=23, srv2117922725=19, srv1173518582=3, srv846353458=28, srv1398041824=6, srv1708278962=9, srv190423615=14, srv312855228=21, srv20786253=18, srv196930741=17, srv110108989=0, srv1801902772=13, srv886417441=29, srv545343646=25, srv1936576159=16, srv1425293750=7, srv1127814338=2, srv1526335495=8, srv1289766122=5, srv1727515349=11, srv727914507=27, srv192126125=15, srv349534445=22, srv470084322=24, srv1728258428=12, srv1216672720=4, srv172236876=10, srv607178195=26, srv1114678074=1, srv233928461=20} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-10 06:53:18,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-10 06:53:18,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-10 06:53:18,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-10 06:53:18,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-10 06:53:18,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-10 06:53:18,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-10 06:53:18,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-10 06:53:18,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-10 06:53:18,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-10 06:53:18,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-10 06:53:18,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-10 06:53:18,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-10 06:53:18,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-10 06:53:18,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-10 06:53:18,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-10 06:53:18,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-10 06:53:18,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-10 06:53:18,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-10 06:53:18,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-10 06:53:18,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-10 06:53:18,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-10 06:53:18,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-10 06:53:18,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-10 06:53:18,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-10 06:53:18,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-10 06:53:18,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-10 06:53:18,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-10 06:53:18,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-10 06:53:18,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-10 06:53:18,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-10 06:53:18,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-12-10 06:53:18,141 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0991366998369543, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-10 06:53:18,142 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912505419768355, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0991366998369543, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-12-10 06:55:18,129 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5613158 different iterations.  Found a solution that moves 1853 regions; Going from a computed imbalance of 0.9912505419768355 to a new imbalance of 4.324994439959734E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6862962962962963, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-10 06:55:18,138 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv110108989:90 , srv1114678074:90 , srv1127814338:90 , srv1173518582:90 , srv1216672720:90 , srv1289766122:90 , srv1398041824:90 , srv1425293750:90 , srv1526335495:90 , srv1708278962:90 , srv172236876:90 , srv1727515349:90 , srv1728258428:90 , srv1801902772:90 , srv190423615:90 , srv192126125:90 , srv1936576159:90 , srv196930741:90 , srv20786253:90 , srv2117922725:90 , srv233928461:90 , srv312855228:90 , srv349534445:90 , srv40377272:90 , srv470084322:90 , srv545343646:90 , srv607178195:90 , srv727914507:90 , srv846353458:90 , srv886417441:90 }
2023-12-10 06:55:18,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-10 06:55:18,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv40377272=23, srv2117922725=19, srv1173518582=3, srv846353458=28, srv1398041824=6, srv1708278962=9, srv190423615=14, srv312855228=21, srv20786253=18, srv196930741=17, srv110108989=0, srv1801902772=13, srv886417441=29, srv545343646=25, srv1936576159=16, srv1425293750=7, srv1127814338=2, srv1526335495=8, srv1289766122=5, srv1727515349=11, srv727914507=27, srv192126125=15, srv349534445=22, srv470084322=24, srv1728258428=12, srv1216672720=4, srv172236876=10, srv607178195=26, srv1114678074=1, srv233928461=20} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-10 06:55:18,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-10 06:55:18,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-10 06:55:18,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-10 06:55:18,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-10 06:55:18,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-10 06:55:18,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-10 06:55:18,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-10 06:55:18,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-10 06:55:18,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-10 06:55:18,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-10 06:55:18,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-10 06:55:18,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-10 06:55:18,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-10 06:55:18,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-10 06:55:18,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-10 06:55:18,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-10 06:55:18,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-10 06:55:18,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-10 06:55:18,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-10 06:55:18,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-10 06:55:18,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-10 06:55:18,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-10 06:55:18,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-10 06:55:18,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-10 06:55:18,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-10 06:55:18,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-10 06:55:18,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-10 06:55:18,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-10 06:55:18,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-10 06:55:18,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-10 06:55:18,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-12-10 06:55:18,165 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-10 06:55:18,165 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv110108989:90 , srv1114678074:90 , srv1127814338:90 , srv1173518582:90 , srv1216672720:90 , srv1289766122:90 , srv1398041824:90 , srv1425293750:90 , srv1526335495:90 , srv1708278962:90 , srv172236876:90 , srv1727515349:90 , srv1728258428:90 , srv1801902772:90 , srv190423615:90 , srv192126125:90 , srv1936576159:90 , srv196930741:90 , srv20786253:90 , srv2117922725:90 , srv233928461:90 , srv312855228:90 , srv349534445:90 , srv40377272:90 , srv470084322:90 , srv545343646:90 , srv607178195:90 , srv727914507:90 , srv846353458:90 , srv886417441:90 }
2023-12-10 06:55:18,185 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=244 (was 311), ProcessCount=171 (was 174), AvailableMemoryMB=5973 (was 6513)