2023-11-28 13:53:16,647 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-11-28 13:53:16,973 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-11-28 13:53:17,089 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-11-28 13:53:17,160 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-28 13:53:17,168 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-11-28 13:53:17,182 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=332, ProcessCount=170, AvailableMemoryMB=7586
2023-11-28 13:53:17,184 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-28 13:53:17,184 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-11-28 13:53:17,306 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1062660050:84 , srv1079757539:84 , srv1086126122:84 , srv1094313417:84 , srv1110702596:84 , srv1157284931:84 , srv119348876:84 , srv1198237756:84 , srv1203686806:84 , srv1244946793:84 , srv1248038747:84 , srv1267741925:84 , srv127496857:84 , srv1296551407:84 , srv1315816931:84 , srv1321399999:84 , srv1391577206:84 , srv1399889201:84 , srv1433881398:84 , srv145365454:84 , srv1471091344:84 , srv1498945653:84 , srv1502956647:84 , srv1518712735:84 , srv1539442668:84 , srv1563537172:84 , srv156494940:84 , srv1579899708:84 , srv1587993699:84 , srv1593018804:84 , srv1594288517:84 , srv1614742248:84 , srv1628272488:84 , srv1647107534:84 , srv1658813877:84 , srv1664941990:84 , srv1667683700:84 , srv1672194606:84 , srv1693818616:84 , srv1720857153:84 , srv1758768233:84 , srv1761769644:84 , srv1772701395:84 , srv1899491117:84 , srv1914150588:84 , srv1927823912:84 , srv1928438512:84 , srv1968727386:84 , srv1978245543:84 , srv1990870751:84 , srv2001488945:84 , srv2019215333:84 , srv2055866089:84 , srv2059675345:84 , srv2075748631:84 , srv208844757:84 , srv2118693399:84 , srv2119958028:84 , srv2122403839:84 , srv2127353330:84 , srv2131988662:84 , srv2142520997:84 , srv244483097:84 , srv284820921:84 , srv307774614:84 , srv334892247:84 , srv337259517:84 , srv353164147:84 , srv372683358:84 , srv420755286:84 , srv446083558:84 , srv461267596:84 , srv490372080:84 , srv511169592:84 , srv520269185:84 , srv546229656:84 , srv557040994:84 , srv580922916:84 , srv60036688:84 , srv608121639:84 , srv62928778:84 , srv634433123:84 , srv646859499:84 , srv650745026:84 , srv660240614:84 , srv671350156:84 , srv675164653:84 , srv704329949:84 , srv753313584:84 , srv762079616:84 , srv774445895:84 , srv796374312:84 , srv812871482:84 , srv81323589:84 , srv816179706:84 , srv915268414:84 , srv930863022:84 , srv93240288:84 , srv99005688:84 , srv1379869307:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-11-28 13:53:17,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-28 13:53:17,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv520269185=75, srv1628272488=33, srv1914150588=45, srv1379869307=16, srv60036688=79, srv1658813877=35, srv1899491117=44, srv461267596=72, srv93240288=98, srv2131988662=61, srv244483097=63, srv1203686806=8, srv1772701395=43, srv307774614=65, srv145365454=20, srv1094313417=3, srv1502956647=23, srv1593018804=30, srv1587993699=29, srv1693818616=39, srv1664941990=36, srv1990870751=50, srv1518712735=24, srv1672194606=38, srv1761769644=42, srv2055866089=53, srv1198237756=7, srv1086126122=2, srv1110702596=4, srv1296551407=13, srv2059675345=54, srv915268414=96, srv1720857153=40, srv337259517=67, srv284820921=64, srv1978245543=49, srv2127353330=60, srv671350156=86, srv1594288517=31, srv1927823912=46, srv446083558=71, srv1315816931=14, srv2075748631=55, srv580922916=78, srv675164653=87, srv1399889201=18, srv812871482=93, srv650745026=84, srv208844757=56, srv1563537172=26, srv930863022=97, srv1157284931=5, srv420755286=70, srv156494940=27, srv99005688=99, srv2119958028=58, srv2122403839=59, srv1244946793=9, srv127496857=12, srv2118693399=57, srv334892247=66, srv1079757539=1, srv1647107534=34, srv1248038747=10, srv660240614=85, srv1062660050=0, srv1321399999=15, srv511169592=74, srv2142520997=62, srv1667683700=37, srv1928438512=47, srv1539442668=25, srv372683358=69, srv816179706=95, srv1267741925=11, srv1471091344=21, srv557040994=77, srv774445895=91, srv1433881398=19, srv2001488945=51, srv704329949=88, srv1498945653=22, srv646859499=83, srv796374312=92, srv762079616=90, srv2019215333=52, srv353164147=68, srv490372080=73, srv81323589=94, srv1579899708=28, srv1968727386=48, srv608121639=80, srv1391577206=17, srv1614742248=32, srv634433123=82, srv119348876=6, srv1758768233=41, srv753313584=89, srv546229656=76, srv62928778=81} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-28 13:53:17,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-28 13:53:17,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-28 13:53:17,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-28 13:53:17,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-28 13:53:17,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-28 13:53:17,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-28 13:53:17,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-28 13:53:17,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-28 13:53:17,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-28 13:53:17,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-28 13:53:17,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-28 13:53:17,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-28 13:53:17,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-28 13:53:17,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-28 13:53:17,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-28 13:53:17,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-28 13:53:17,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-28 13:53:17,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-28 13:53:17,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-28 13:53:17,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-28 13:53:17,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-28 13:53:17,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-28 13:53:17,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-28 13:53:17,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-28 13:53:17,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-28 13:53:17,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-28 13:53:17,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-28 13:53:17,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-28 13:53:17,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-28 13:53:17,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-28 13:53:17,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-28 13:53:17,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-28 13:53:17,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-28 13:53:17,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-28 13:53:17,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-28 13:53:17,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-28 13:53:17,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-28 13:53:17,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-28 13:53:17,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-28 13:53:17,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-28 13:53:17,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-28 13:53:17,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-28 13:53:17,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-28 13:53:17,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-28 13:53:17,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-28 13:53:17,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-28 13:53:17,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-28 13:53:17,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-28 13:53:17,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-28 13:53:17,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-28 13:53:17,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-28 13:53:17,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-28 13:53:17,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-28 13:53:17,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-28 13:53:17,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-28 13:53:17,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-28 13:53:17,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-28 13:53:17,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-28 13:53:17,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-28 13:53:17,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-28 13:53:17,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-28 13:53:17,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-28 13:53:17,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-28 13:53:17,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-28 13:53:17,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-28 13:53:17,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-28 13:53:17,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-28 13:53:17,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-28 13:53:17,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-28 13:53:17,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-28 13:53:17,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-28 13:53:17,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-28 13:53:17,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-28 13:53:17,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-28 13:53:17,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-28 13:53:17,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-28 13:53:17,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-28 13:53:17,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-28 13:53:17,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-28 13:53:17,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-28 13:53:17,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-28 13:53:17,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-28 13:53:17,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-28 13:53:17,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-28 13:53:17,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-28 13:53:17,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-28 13:53:17,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-28 13:53:17,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-28 13:53:17,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-28 13:53:17,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-28 13:53:17,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-28 13:53:17,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-28 13:53:17,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-28 13:53:17,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-28 13:53:17,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-28 13:53:17,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-28 13:53:17,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-28 13:53:17,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-28 13:53:17,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-28 13:53:17,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-28 13:53:17,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-28 13:53:17,737 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-28 13:53:17,737 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-28 13:53:17,738 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-28 13:53:18,206 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 775 ms to try 5000 different iterations.  Found a solution that moves 4689 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18589542166900766. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04063377448479782, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05497270009132795, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.521, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04063377448479782, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.18083333333333335, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.25125, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-28 13:53:18,217 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1086126122:76 , srv1321399999:80 , srv1758768233:81 , srv646859499:81 , srv1094313417:82 , srv1244946793:82 , srv1399889201:82 , srv2127353330:82 , srv99005688:82 , srv1248038747:83 , srv1978245543:83 , srv2142520997:83 , srv1203686806:84 , srv1315816931:84 , srv1471091344:84 , srv156494940:84 , srv1594288517:84 , srv1772701395:84 , srv1914150588:84 , srv1928438512:84 , srv2118693399:84 , srv660240614:84 , srv1157284931:85 , srv145365454:85 , srv1518712735:85 , srv1761769644:85 , srv1927823912:85 , srv208844757:85 , srv2131988662:85 , srv244483097:85 , srv372683358:85 , srv81323589:85 , srv119348876:86 , srv1296551407:86 , srv1587993699:86 , srv1628272488:86 , srv1693818616:86 , srv2055866089:86 , srv2119958028:86 , srv446083558:86 , srv461267596:86 , srv490372080:86 , srv511169592:86 , srv557040994:86 , srv608121639:86 , srv650745026:86 , srv753313584:86 , srv774445895:86 , srv812871482:86 , srv816179706:86 , srv915268414:86 , srv930863022:86 , srv93240288:86 , srv1267741925:87 , srv1502956647:87 , srv1563537172:87 , srv1579899708:87 , srv1990870751:87 , srv2001488945:87 , srv2019215333:87 , srv2059675345:87 , srv520269185:87 , srv546229656:87 , srv580922916:87 , srv634433123:87 , srv704329949:87 , srv762079616:87 , srv1198237756:88 , srv127496857:88 , srv1433881398:88 , srv1647107534:88 , srv1658813877:88 , srv1968727386:88 , srv2075748631:88 , srv284820921:88 , srv307774614:88 , srv334892247:88 , srv420755286:88 , srv796374312:88 , srv1498945653:89 , srv1539442668:89 , srv1614742248:89 , srv1667683700:89 , srv1672194606:89 , srv1720857153:89 , srv337259517:89 , srv60036688:89 , srv671350156:89 , srv675164653:89 , srv1664941990:90 , srv1899491117:90 , srv2122403839:90 , srv1062660050:91 , srv1391577206:91 , srv1593018804:91 , srv62928778:91 , srv1079757539:92 , srv1110702596:93 , srv353164147:93 , srv1379869307:451 }
2023-11-28 13:53:18,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-28 13:53:18,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv520269185=75, srv1628272488=33, srv1914150588=45, srv1379869307=16, srv60036688=79, srv1658813877=35, srv1899491117=44, srv461267596=72, srv93240288=98, srv2131988662=61, srv244483097=63, srv1203686806=8, srv1772701395=43, srv307774614=65, srv145365454=20, srv1094313417=3, srv1502956647=23, srv1593018804=30, srv1587993699=29, srv1693818616=39, srv1664941990=36, srv1990870751=50, srv1518712735=24, srv1672194606=38, srv1761769644=42, srv2055866089=53, srv1198237756=7, srv1086126122=2, srv1110702596=4, srv1296551407=13, srv2059675345=54, srv915268414=96, srv1720857153=40, srv337259517=67, srv284820921=64, srv1978245543=49, srv2127353330=60, srv671350156=86, srv1594288517=31, srv1927823912=46, srv446083558=71, srv1315816931=14, srv2075748631=55, srv580922916=78, srv675164653=87, srv1399889201=18, srv812871482=93, srv650745026=84, srv208844757=56, srv1563537172=26, srv930863022=97, srv1157284931=5, srv420755286=70, srv156494940=27, srv99005688=99, srv2119958028=58, srv2122403839=59, srv1244946793=9, srv127496857=12, srv2118693399=57, srv334892247=66, srv1079757539=1, srv1647107534=34, srv1248038747=10, srv660240614=85, srv1062660050=0, srv1321399999=15, srv511169592=74, srv2142520997=62, srv1667683700=37, srv1928438512=47, srv1539442668=25, srv372683358=69, srv816179706=95, srv1267741925=11, srv1471091344=21, srv557040994=77, srv774445895=91, srv1433881398=19, srv2001488945=51, srv704329949=88, srv1498945653=22, srv646859499=83, srv796374312=92, srv762079616=90, srv2019215333=52, srv353164147=68, srv490372080=73, srv81323589=94, srv1579899708=28, srv1968727386=48, srv608121639=80, srv1391577206=17, srv1614742248=32, srv634433123=82, srv119348876=6, srv1758768233=41, srv753313584=89, srv546229656=76, srv62928778=81} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-28 13:53:18,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-28 13:53:18,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-28 13:53:18,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-28 13:53:18,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-28 13:53:18,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-28 13:53:18,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-28 13:53:18,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-28 13:53:18,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-28 13:53:18,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-28 13:53:18,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-28 13:53:18,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-28 13:53:18,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-28 13:53:18,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-28 13:53:18,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-28 13:53:18,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-28 13:53:18,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-28 13:53:18,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-28 13:53:18,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-28 13:53:18,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-28 13:53:18,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-28 13:53:18,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-28 13:53:18,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-28 13:53:18,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-28 13:53:18,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-28 13:53:18,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-28 13:53:18,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-28 13:53:18,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-28 13:53:18,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-28 13:53:18,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-28 13:53:18,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-28 13:53:18,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-28 13:53:18,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-28 13:53:18,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-28 13:53:18,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-28 13:53:18,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-28 13:53:18,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-28 13:53:18,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-28 13:53:18,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-28 13:53:18,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-28 13:53:18,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-28 13:53:18,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-28 13:53:18,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-28 13:53:18,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-28 13:53:18,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-28 13:53:18,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-28 13:53:18,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-28 13:53:18,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-28 13:53:18,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-28 13:53:18,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-28 13:53:18,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-28 13:53:18,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-28 13:53:18,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-28 13:53:18,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-28 13:53:18,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-28 13:53:18,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-28 13:53:18,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-28 13:53:18,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-28 13:53:18,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-28 13:53:18,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-28 13:53:18,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-28 13:53:18,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-28 13:53:18,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-28 13:53:18,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-28 13:53:18,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-28 13:53:18,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-28 13:53:18,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-28 13:53:18,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-28 13:53:18,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-28 13:53:18,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-28 13:53:18,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-28 13:53:18,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-28 13:53:18,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-28 13:53:18,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-28 13:53:18,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-28 13:53:18,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-28 13:53:18,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-28 13:53:18,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-28 13:53:18,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-28 13:53:18,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-28 13:53:18,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-28 13:53:18,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-28 13:53:18,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-28 13:53:18,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-28 13:53:18,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-28 13:53:18,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-28 13:53:18,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-28 13:53:18,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-28 13:53:18,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-28 13:53:18,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-28 13:53:18,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-28 13:53:18,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-28 13:53:18,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-28 13:53:18,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-28 13:53:18,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-28 13:53:18,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-28 13:53:18,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-28 13:53:18,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-28 13:53:18,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-28 13:53:18,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-28 13:53:18,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-28 13:53:18,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-28 13:53:18,411 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04063377448479782, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05497270009132795, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04063377448479782, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.18083333333333335, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.25125, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-28 13:53:18,411 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-28 13:53:18,411 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.1858625885892522, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04063377448479782, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05497270009132795, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04063377448479782, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.18083333333333335, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.25125, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-28 13:53:18,763 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 487 ms to try 5000 different iterations.  Found a solution that moves 2656 regions; Going from a computed imbalance of 0.1858625885892522 to a new imbalance of 0.01829650923191. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.019943804328378886); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.034171285718813206, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2951111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.019943804328378886); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.015); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.05025, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-28 13:53:18,774 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1079757539:77 , srv1110702596:79 , srv1593018804:79 , srv1391577206:80 , srv1672194606:80 , srv1664941990:81 , srv1899491117:81 , srv1062660050:82 , srv1198237756:82 , srv127496857:82 , srv1433881398:82 , srv1579899708:82 , srv2122403839:82 , srv62928778:82 , srv796374312:82 , srv1539442668:83 , srv1667683700:83 , srv2075748631:83 , srv334892247:83 , srv353164147:83 , srv511169592:83 , srv671350156:83 , srv675164653:83 , srv704329949:83 , srv762079616:83 , srv1296551407:84 , srv1563537172:84 , srv1693818616:84 , srv1990870751:84 , srv634433123:84 , srv812871482:84 , srv1267741925:85 , srv1614742248:85 , srv1647107534:85 , srv1658813877:85 , srv1720857153:85 , srv1968727386:85 , srv2019215333:85 , srv307774614:85 , srv337259517:85 , srv420755286:85 , srv1157284931:86 , srv119348876:86 , srv1498945653:86 , srv1927823912:86 , srv2001488945:86 , srv2055866089:86 , srv2059675345:86 , srv2131988662:86 , srv284820921:86 , srv461267596:86 , srv490372080:86 , srv580922916:86 , srv608121639:86 , srv650745026:86 , srv753313584:86 , srv774445895:86 , srv816179706:86 , srv915268414:86 , srv1315816931:87 , srv1502956647:87 , srv1518712735:87 , srv1594288517:87 , srv1761769644:87 , srv1772701395:87 , srv446083558:87 , srv520269185:87 , srv546229656:87 , srv557040994:87 , srv660240614:87 , srv930863022:87 , srv93240288:87 , srv1471091344:88 , srv1587993699:88 , srv1914150588:88 , srv208844757:88 , srv2119958028:88 , srv2142520997:88 , srv244483097:88 , srv60036688:88 , srv1203686806:89 , srv1244946793:89 , srv145365454:89 , srv1628272488:89 , srv1978245543:89 , srv2118693399:89 , srv372683358:89 , srv81323589:89 , srv1094313417:90 , srv1248038747:90 , srv156494940:90 , srv1928438512:90 , srv2127353330:90 , srv1321399999:91 , srv646859499:91 , srv99005688:91 , srv1758768233:92 , srv1399889201:94 , srv1086126122:99 , srv1379869307:500 }
2023-11-28 13:53:18,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-28 13:53:18,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv520269185=75, srv1628272488=33, srv1914150588=45, srv1379869307=16, srv60036688=79, srv1658813877=35, srv1899491117=44, srv461267596=72, srv93240288=98, srv2131988662=61, srv244483097=63, srv1203686806=8, srv1772701395=43, srv307774614=65, srv145365454=20, srv1094313417=3, srv1502956647=23, srv1593018804=30, srv1587993699=29, srv1693818616=39, srv1664941990=36, srv1990870751=50, srv1518712735=24, srv1672194606=38, srv1761769644=42, srv2055866089=53, srv1198237756=7, srv1086126122=2, srv1110702596=4, srv1296551407=13, srv2059675345=54, srv915268414=96, srv1720857153=40, srv337259517=67, srv284820921=64, srv1978245543=49, srv2127353330=60, srv671350156=86, srv1594288517=31, srv1927823912=46, srv446083558=71, srv1315816931=14, srv2075748631=55, srv580922916=78, srv675164653=87, srv1399889201=18, srv812871482=93, srv650745026=84, srv208844757=56, srv1563537172=26, srv930863022=97, srv1157284931=5, srv420755286=70, srv156494940=27, srv99005688=99, srv2119958028=58, srv2122403839=59, srv1244946793=9, srv127496857=12, srv2118693399=57, srv334892247=66, srv1079757539=1, srv1647107534=34, srv1248038747=10, srv660240614=85, srv1062660050=0, srv1321399999=15, srv511169592=74, srv2142520997=62, srv1667683700=37, srv1928438512=47, srv1539442668=25, srv372683358=69, srv816179706=95, srv1267741925=11, srv1471091344=21, srv557040994=77, srv774445895=91, srv1433881398=19, srv2001488945=51, srv704329949=88, srv1498945653=22, srv646859499=83, srv796374312=92, srv762079616=90, srv2019215333=52, srv353164147=68, srv490372080=73, srv81323589=94, srv1579899708=28, srv1968727386=48, srv608121639=80, srv1391577206=17, srv1614742248=32, srv634433123=82, srv119348876=6, srv1758768233=41, srv753313584=89, srv546229656=76, srv62928778=81} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-28 13:53:18,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-28 13:53:18,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-28 13:53:18,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-28 13:53:18,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-28 13:53:18,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-28 13:53:18,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-28 13:53:18,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-28 13:53:18,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-28 13:53:18,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-28 13:53:18,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-28 13:53:18,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-28 13:53:18,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-28 13:53:18,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-28 13:53:18,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-28 13:53:18,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-28 13:53:18,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-28 13:53:18,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-28 13:53:18,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-28 13:53:18,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-28 13:53:18,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-28 13:53:18,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-28 13:53:18,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-28 13:53:18,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-28 13:53:18,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-28 13:53:18,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-28 13:53:18,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-28 13:53:18,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-28 13:53:18,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-28 13:53:18,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-28 13:53:18,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-28 13:53:18,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-28 13:53:18,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-28 13:53:18,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-28 13:53:18,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-28 13:53:18,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-28 13:53:18,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-28 13:53:18,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-28 13:53:18,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-28 13:53:18,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-28 13:53:18,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-28 13:53:18,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-28 13:53:18,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-28 13:53:18,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-28 13:53:18,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-28 13:53:18,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-28 13:53:18,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-28 13:53:18,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-28 13:53:18,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-28 13:53:18,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-28 13:53:18,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-28 13:53:18,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-28 13:53:18,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-28 13:53:18,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-28 13:53:18,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-28 13:53:18,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-28 13:53:18,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-28 13:53:18,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-28 13:53:18,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-28 13:53:18,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-28 13:53:18,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-28 13:53:18,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-28 13:53:18,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-28 13:53:18,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-28 13:53:18,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-28 13:53:18,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-28 13:53:18,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-28 13:53:18,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-28 13:53:18,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-28 13:53:18,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-28 13:53:18,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-28 13:53:18,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-28 13:53:18,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-28 13:53:18,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-28 13:53:18,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-28 13:53:18,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-28 13:53:18,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-28 13:53:18,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-28 13:53:18,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-28 13:53:18,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-28 13:53:18,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-28 13:53:18,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-28 13:53:18,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-28 13:53:18,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-28 13:53:18,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-28 13:53:18,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-28 13:53:18,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-28 13:53:18,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-28 13:53:18,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-28 13:53:18,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-28 13:53:18,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-28 13:53:18,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-28 13:53:18,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-28 13:53:18,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-28 13:53:18,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-28 13:53:18,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-28 13:53:18,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-28 13:53:18,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-28 13:53:18,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-28 13:53:18,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-28 13:53:18,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-28 13:53:18,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-28 13:53:18,970 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.019943804328378886); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.034171285718813206, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.019943804328378886); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.015); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.05025, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-28 13:53:18,970 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-28 13:53:18,970 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.018277911522413186, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.019943804328378886); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.034171285718813206, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.019943804328378886); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.015); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.05025, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-28 13:53:19,251 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 408 ms to try 5000 different iterations.  Found a solution that moves 1033 regions; Going from a computed imbalance of 0.018277911522413186 to a new imbalance of 7.163536411623038E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005538694953529477); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008274200976271674); MoveCostFunction : (multiplier=7.0, imbalance=0.11477777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005538694953529477); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-28 13:53:19,253 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv60036688:80 , srv1399889201:81 , srv353164147:81 , srv1628272488:82 , srv1086126122:83 , srv1498945653:83 , srv156494940:83 , srv1587993699:83 , srv1614742248:83 , srv1928438512:83 , srv546229656:83 , srv81323589:83 , srv1062660050:84 , srv119348876:84 , srv1203686806:84 , srv1248038747:84 , srv145365454:84 , srv1502956647:84 , srv1720857153:84 , srv1758768233:84 , srv2001488945:84 , srv208844757:84 , srv2118693399:84 , srv2119958028:84 , srv244483097:84 , srv284820921:84 , srv337259517:84 , srv372683358:84 , srv420755286:84 , srv446083558:84 , srv520269185:84 , srv580922916:84 , srv62928778:84 , srv93240288:84 , srv99005688:84 , srv1110702596:85 , srv1267741925:85 , srv1471091344:85 , srv1518712735:85 , srv1647107534:85 , srv1658813877:85 , srv1667683700:85 , srv1968727386:85 , srv2019215333:85 , srv2059675345:85 , srv2122403839:85 , srv2127353330:85 , srv307774614:85 , srv461267596:85 , srv557040994:85 , srv608121639:85 , srv650745026:85 , srv675164653:85 , srv774445895:85 , srv816179706:85 , srv930863022:85 , srv1094313417:86 , srv1539442668:86 , srv1563537172:86 , srv1594288517:86 , srv1761769644:86 , srv1914150588:86 , srv1927823912:86 , srv2055866089:86 , srv2075748631:86 , srv2142520997:86 , srv334892247:86 , srv490372080:86 , srv646859499:86 , srv671350156:86 , srv753313584:86 , srv915268414:86 , srv1157284931:87 , srv1198237756:87 , srv1244946793:87 , srv1296551407:87 , srv1315816931:87 , srv1321399999:87 , srv1391577206:87 , srv1664941990:87 , srv1693818616:87 , srv1772701395:87 , srv1899491117:87 , srv1978245543:87 , srv2131988662:87 , srv634433123:87 , srv660240614:87 , srv704329949:87 , srv796374312:87 , srv1433881398:88 , srv1579899708:88 , srv1990870751:88 , srv511169592:88 , srv762079616:88 , srv812871482:88 , srv1079757539:89 , srv127496857:89 , srv1593018804:89 , srv1672194606:89 , srv1379869307:556 }
2023-11-28 13:53:19,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-28 13:53:19,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv520269185=75, srv1628272488=33, srv1914150588=45, srv1379869307=16, srv60036688=79, srv1658813877=35, srv1899491117=44, srv461267596=72, srv93240288=98, srv2131988662=61, srv244483097=63, srv1203686806=8, srv1772701395=43, srv307774614=65, srv145365454=20, srv1094313417=3, srv1502956647=23, srv1593018804=30, srv1587993699=29, srv1693818616=39, srv1664941990=36, srv1990870751=50, srv1518712735=24, srv1672194606=38, srv1761769644=42, srv2055866089=53, srv1198237756=7, srv1086126122=2, srv1110702596=4, srv1296551407=13, srv2059675345=54, srv915268414=96, srv1720857153=40, srv337259517=67, srv284820921=64, srv1978245543=49, srv2127353330=60, srv671350156=86, srv1594288517=31, srv1927823912=46, srv446083558=71, srv1315816931=14, srv2075748631=55, srv580922916=78, srv675164653=87, srv1399889201=18, srv812871482=93, srv650745026=84, srv208844757=56, srv1563537172=26, srv930863022=97, srv1157284931=5, srv420755286=70, srv156494940=27, srv99005688=99, srv2119958028=58, srv2122403839=59, srv1244946793=9, srv127496857=12, srv2118693399=57, srv334892247=66, srv1079757539=1, srv1647107534=34, srv1248038747=10, srv660240614=85, srv1062660050=0, srv1321399999=15, srv511169592=74, srv2142520997=62, srv1667683700=37, srv1928438512=47, srv1539442668=25, srv372683358=69, srv816179706=95, srv1267741925=11, srv1471091344=21, srv557040994=77, srv774445895=91, srv1433881398=19, srv2001488945=51, srv704329949=88, srv1498945653=22, srv646859499=83, srv796374312=92, srv762079616=90, srv2019215333=52, srv353164147=68, srv490372080=73, srv81323589=94, srv1579899708=28, srv1968727386=48, srv608121639=80, srv1391577206=17, srv1614742248=32, srv634433123=82, srv119348876=6, srv1758768233=41, srv753313584=89, srv546229656=76, srv62928778=81} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-28 13:53:19,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-28 13:53:19,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-28 13:53:19,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-28 13:53:19,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-28 13:53:19,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-28 13:53:19,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-28 13:53:19,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-28 13:53:19,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-28 13:53:19,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-28 13:53:19,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-28 13:53:19,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-28 13:53:19,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-28 13:53:19,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-28 13:53:19,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-28 13:53:19,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-28 13:53:19,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-28 13:53:19,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-28 13:53:19,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-28 13:53:19,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-28 13:53:19,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-28 13:53:19,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-28 13:53:19,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-28 13:53:19,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-28 13:53:19,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-28 13:53:19,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-28 13:53:19,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-28 13:53:19,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-28 13:53:19,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-28 13:53:19,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-28 13:53:19,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-28 13:53:19,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-28 13:53:19,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-28 13:53:19,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-28 13:53:19,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-28 13:53:19,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-28 13:53:19,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-28 13:53:19,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-28 13:53:19,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-28 13:53:19,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-28 13:53:19,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-28 13:53:19,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-28 13:53:19,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-28 13:53:19,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-28 13:53:19,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-28 13:53:19,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-28 13:53:19,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-28 13:53:19,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-28 13:53:19,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-28 13:53:19,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-28 13:53:19,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-28 13:53:19,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-28 13:53:19,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-28 13:53:19,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-28 13:53:19,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-28 13:53:19,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-28 13:53:19,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-28 13:53:19,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-28 13:53:19,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-28 13:53:19,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-28 13:53:19,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-28 13:53:19,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-28 13:53:19,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-28 13:53:19,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-28 13:53:19,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-28 13:53:19,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-28 13:53:19,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-28 13:53:19,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-28 13:53:19,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-28 13:53:19,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-28 13:53:19,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-28 13:53:19,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-28 13:53:19,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-28 13:53:19,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-28 13:53:19,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-28 13:53:19,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-28 13:53:19,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-28 13:53:19,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-28 13:53:19,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-28 13:53:19,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-28 13:53:19,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-28 13:53:19,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-28 13:53:19,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-28 13:53:19,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-28 13:53:19,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-28 13:53:19,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-28 13:53:19,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-28 13:53:19,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-28 13:53:19,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-28 13:53:19,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-28 13:53:19,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-28 13:53:19,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-28 13:53:19,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-28 13:53:19,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-28 13:53:19,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-28 13:53:19,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-28 13:53:19,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-28 13:53:19,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-28 13:53:19,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-28 13:53:19,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-28 13:53:19,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-28 13:53:19,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-28 13:53:19,441 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005538694953529477); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008274200976271674); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005538694953529477); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-28 13:53:19,441 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-28 13:53:19,441 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=7.0912042056358E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005538694953529477); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008274200976271674); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005538694953529477); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-28 13:53:19,936 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 628 ms to try 5000 different iterations.  Found a solution that moves 226 regions; Going from a computed imbalance of 7.0912042056358E-4 to a new imbalance of 1.171361735735808E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017727200782610348); MoveCostFunction : (multiplier=7.0, imbalance=0.025111111111111112, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-28 13:53:19,937 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv127496857:82 , srv1593018804:83 , srv1978245543:83 , srv1990870751:83 , srv915268414:83 , srv1079757539:84 , srv1086126122:84 , srv1094313417:84 , srv1157284931:84 , srv1203686806:84 , srv1244946793:84 , srv1315816931:84 , srv1321399999:84 , srv1391577206:84 , srv1433881398:84 , srv145365454:84 , srv1498945653:84 , srv1502956647:84 , srv1539442668:84 , srv1579899708:84 , srv1647107534:84 , srv1658813877:84 , srv1664941990:84 , srv1672194606:84 , srv1720857153:84 , srv1761769644:84 , srv1772701395:84 , srv1899491117:84 , srv1914150588:84 , srv1968727386:84 , srv2055866089:84 , srv2059675345:84 , srv2119958028:84 , srv2131988662:84 , srv284820921:84 , srv307774614:84 , srv337259517:84 , srv372683358:84 , srv490372080:84 , srv511169592:84 , srv520269185:84 , srv546229656:84 , srv557040994:84 , srv580922916:84 , srv646859499:84 , srv660240614:84 , srv753313584:84 , srv762079616:84 , srv796374312:84 , srv812871482:84 , srv930863022:84 , srv1062660050:85 , srv1110702596:85 , srv1248038747:85 , srv1267741925:85 , srv1296551407:85 , srv1399889201:85 , srv1471091344:85 , srv1518712735:85 , srv156494940:85 , srv1587993699:85 , srv1594288517:85 , srv1614742248:85 , srv1628272488:85 , srv1667683700:85 , srv1693818616:85 , srv1927823912:85 , srv1928438512:85 , srv2001488945:85 , srv2019215333:85 , srv2075748631:85 , srv208844757:85 , srv2118693399:85 , srv2122403839:85 , srv2127353330:85 , srv244483097:85 , srv334892247:85 , srv420755286:85 , srv446083558:85 , srv461267596:85 , srv60036688:85 , srv608121639:85 , srv62928778:85 , srv634433123:85 , srv650745026:85 , srv671350156:85 , srv675164653:85 , srv704329949:85 , srv774445895:85 , srv81323589:85 , srv816179706:85 , srv93240288:85 , srv99005688:85 , srv119348876:86 , srv1198237756:86 , srv1563537172:86 , srv1758768233:86 , srv2142520997:86 , srv353164147:86 , srv1379869307:636 }
2023-11-28 13:53:19,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-28 13:53:19,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv520269185=75, srv1628272488=33, srv1914150588=45, srv1379869307=16, srv60036688=79, srv1658813877=35, srv1899491117=44, srv461267596=72, srv93240288=98, srv2131988662=61, srv244483097=63, srv1203686806=8, srv1772701395=43, srv307774614=65, srv145365454=20, srv1094313417=3, srv1502956647=23, srv1593018804=30, srv1587993699=29, srv1693818616=39, srv1664941990=36, srv1990870751=50, srv1518712735=24, srv1672194606=38, srv1761769644=42, srv2055866089=53, srv1198237756=7, srv1086126122=2, srv1110702596=4, srv1296551407=13, srv2059675345=54, srv915268414=96, srv1720857153=40, srv337259517=67, srv284820921=64, srv1978245543=49, srv2127353330=60, srv671350156=86, srv1594288517=31, srv1927823912=46, srv446083558=71, srv1315816931=14, srv2075748631=55, srv580922916=78, srv675164653=87, srv1399889201=18, srv812871482=93, srv650745026=84, srv208844757=56, srv1563537172=26, srv930863022=97, srv1157284931=5, srv420755286=70, srv156494940=27, srv99005688=99, srv2119958028=58, srv2122403839=59, srv1244946793=9, srv127496857=12, srv2118693399=57, srv334892247=66, srv1079757539=1, srv1647107534=34, srv1248038747=10, srv660240614=85, srv1062660050=0, srv1321399999=15, srv511169592=74, srv2142520997=62, srv1667683700=37, srv1928438512=47, srv1539442668=25, srv372683358=69, srv816179706=95, srv1267741925=11, srv1471091344=21, srv557040994=77, srv774445895=91, srv1433881398=19, srv2001488945=51, srv704329949=88, srv1498945653=22, srv646859499=83, srv796374312=92, srv762079616=90, srv2019215333=52, srv353164147=68, srv490372080=73, srv81323589=94, srv1579899708=28, srv1968727386=48, srv608121639=80, srv1391577206=17, srv1614742248=32, srv634433123=82, srv119348876=6, srv1758768233=41, srv753313584=89, srv546229656=76, srv62928778=81} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-28 13:53:19,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-28 13:53:19,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-28 13:53:19,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-28 13:53:19,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-28 13:53:19,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-28 13:53:19,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-28 13:53:19,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-28 13:53:19,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-28 13:53:19,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-28 13:53:19,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-28 13:53:19,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-28 13:53:19,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-28 13:53:19,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-28 13:53:19,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-28 13:53:19,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-28 13:53:19,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-28 13:53:19,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-28 13:53:19,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-28 13:53:19,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-28 13:53:19,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-28 13:53:19,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-28 13:53:19,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-28 13:53:19,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-28 13:53:19,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-28 13:53:19,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-28 13:53:19,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-28 13:53:19,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-28 13:53:19,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-28 13:53:19,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-28 13:53:19,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-28 13:53:19,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-28 13:53:19,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-28 13:53:19,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-28 13:53:19,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-28 13:53:19,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-28 13:53:19,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-28 13:53:19,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-28 13:53:19,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-28 13:53:19,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-28 13:53:19,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-28 13:53:19,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-28 13:53:19,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-28 13:53:19,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-28 13:53:19,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-28 13:53:19,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-28 13:53:19,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-28 13:53:19,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-28 13:53:19,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-28 13:53:19,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-28 13:53:19,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-28 13:53:19,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-28 13:53:19,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-28 13:53:19,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-28 13:53:19,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-28 13:53:19,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-28 13:53:19,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-28 13:53:19,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-28 13:53:19,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-28 13:53:19,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-28 13:53:19,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-28 13:53:19,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-28 13:53:19,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-28 13:53:19,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-28 13:53:19,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-28 13:53:19,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-28 13:53:19,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-28 13:53:19,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-28 13:53:19,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-28 13:53:19,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-28 13:53:19,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-28 13:53:19,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-28 13:53:19,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-28 13:53:19,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-28 13:53:19,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-28 13:53:19,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-28 13:53:19,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-28 13:53:19,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-28 13:53:19,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-28 13:53:19,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-28 13:53:19,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-28 13:53:19,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-28 13:53:19,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-28 13:53:19,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-28 13:53:19,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-28 13:53:19,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-28 13:53:19,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-28 13:53:19,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-28 13:53:19,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-28 13:53:19,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-28 13:53:19,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-28 13:53:19,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-28 13:53:19,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-28 13:53:19,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-28 13:53:19,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-28 13:53:19,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-28 13:53:19,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-28 13:53:19,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-28 13:53:19,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-28 13:53:19,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-28 13:53:19,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-28 13:53:19,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-28 13:53:20,121 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.0131131534210373E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017727200782610348); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-28 13:53:20,122 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv127496857:82 , srv1593018804:83 , srv1978245543:83 , srv1990870751:83 , srv915268414:83 , srv1079757539:84 , srv1086126122:84 , srv1094313417:84 , srv1157284931:84 , srv1203686806:84 , srv1244946793:84 , srv1315816931:84 , srv1321399999:84 , srv1391577206:84 , srv1433881398:84 , srv145365454:84 , srv1498945653:84 , srv1502956647:84 , srv1539442668:84 , srv1579899708:84 , srv1647107534:84 , srv1658813877:84 , srv1664941990:84 , srv1672194606:84 , srv1720857153:84 , srv1761769644:84 , srv1772701395:84 , srv1899491117:84 , srv1914150588:84 , srv1968727386:84 , srv2055866089:84 , srv2059675345:84 , srv2119958028:84 , srv2131988662:84 , srv284820921:84 , srv307774614:84 , srv337259517:84 , srv372683358:84 , srv490372080:84 , srv511169592:84 , srv520269185:84 , srv546229656:84 , srv557040994:84 , srv580922916:84 , srv646859499:84 , srv660240614:84 , srv753313584:84 , srv762079616:84 , srv796374312:84 , srv812871482:84 , srv930863022:84 , srv1062660050:85 , srv1110702596:85 , srv1248038747:85 , srv1267741925:85 , srv1296551407:85 , srv1399889201:85 , srv1471091344:85 , srv1518712735:85 , srv156494940:85 , srv1587993699:85 , srv1594288517:85 , srv1614742248:85 , srv1628272488:85 , srv1667683700:85 , srv1693818616:85 , srv1927823912:85 , srv1928438512:85 , srv2001488945:85 , srv2019215333:85 , srv2075748631:85 , srv208844757:85 , srv2118693399:85 , srv2122403839:85 , srv2127353330:85 , srv244483097:85 , srv334892247:85 , srv420755286:85 , srv446083558:85 , srv461267596:85 , srv60036688:85 , srv608121639:85 , srv62928778:85 , srv634433123:85 , srv650745026:85 , srv671350156:85 , srv675164653:85 , srv704329949:85 , srv774445895:85 , srv81323589:85 , srv816179706:85 , srv93240288:85 , srv99005688:85 , srv119348876:86 , srv1198237756:86 , srv1563537172:86 , srv1758768233:86 , srv2142520997:86 , srv353164147:86 , srv1379869307:636 }
2023-11-28 13:53:20,156 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=361 (was 332) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=6708 (was 7586)
2023-11-28 13:53:20,163 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=361, ProcessCount=170, AvailableMemoryMB=6710
2023-11-28 13:53:20,164 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-28 13:53:20,165 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-11-28 13:53:20,180 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1230394:84 , srv1271048017:84 , srv1348727905:84 , srv1426890452:84 , srv1516054427:84 , srv1558536274:84 , srv1561509134:84 , srv1572421028:84 , srv1627978022:84 , srv1630282758:84 , srv1787123122:84 , srv182346811:84 , srv1835584236:84 , srv1860857775:84 , srv1979007883:84 , srv198474413:84 , srv200100457:84 , srv225384920:84 , srv263650571:84 , srv321522119:84 , srv352722793:84 , srv43912513:84 , srv472997169:84 , srv565982095:84 , srv567872626:84 , srv852666253:84 , srv904893914:84 , srv926021086:84 , srv981069433:84 , srv976100583:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-11-28 13:53:20,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-28 13:53:20,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1627978022=8, srv1630282758=9, srv926021086=27, srv1787123122=10, srv1835584236=12, srv472997169=22, srv1271048017=1, srv565982095=23, srv43912513=21, srv263650571=18, srv981069433=29, srv182346811=11, srv1558536274=5, srv352722793=20, srv852666253=25, srv1572421028=7, srv1979007883=14, srv976100583=28, srv567872626=24, srv321522119=19, srv1426890452=3, srv198474413=15, srv225384920=17, srv904893914=26, srv1230394=0, srv1860857775=13, srv1561509134=6, srv200100457=16, srv1516054427=4, srv1348727905=2} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-28 13:53:20,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-28 13:53:20,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-28 13:53:20,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-28 13:53:20,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-28 13:53:20,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-28 13:53:20,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-28 13:53:20,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-28 13:53:20,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-28 13:53:20,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-28 13:53:20,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-28 13:53:20,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-28 13:53:20,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-28 13:53:20,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-28 13:53:20,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-28 13:53:20,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-28 13:53:20,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-28 13:53:20,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-28 13:53:20,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-28 13:53:20,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-28 13:53:20,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-28 13:53:20,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-28 13:53:20,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-28 13:53:20,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-28 13:53:20,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-28 13:53:20,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-28 13:53:20,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-28 13:53:20,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-28 13:53:20,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-28 13:53:20,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-28 13:53:20,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-28 13:53:20,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-11-28 13:53:20,214 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1613159120945078, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-28 13:53:20,214 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912701344435839, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1613159120945078, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-11-28 13:55:20,197 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5822735 different iterations.  Found a solution that moves 1856 regions; Going from a computed imbalance of 0.9912701344435839 to a new imbalance of 4.3319965896196796E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6874074074074074, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-28 13:55:20,203 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1230394:90 , srv1271048017:90 , srv1348727905:90 , srv1426890452:90 , srv1516054427:90 , srv1558536274:90 , srv1561509134:90 , srv1572421028:90 , srv1627978022:90 , srv1630282758:90 , srv1787123122:90 , srv182346811:90 , srv1835584236:90 , srv1860857775:90 , srv1979007883:90 , srv198474413:90 , srv200100457:90 , srv225384920:90 , srv263650571:90 , srv321522119:90 , srv352722793:90 , srv43912513:90 , srv472997169:90 , srv565982095:90 , srv567872626:90 , srv852666253:90 , srv904893914:90 , srv926021086:90 , srv976100583:90 , srv981069433:90 }
2023-11-28 13:55:20,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-28 13:55:20,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1627978022=8, srv1630282758=9, srv926021086=27, srv1787123122=10, srv1835584236=12, srv472997169=22, srv1271048017=1, srv565982095=23, srv43912513=21, srv263650571=18, srv981069433=29, srv182346811=11, srv1558536274=5, srv352722793=20, srv852666253=25, srv1572421028=7, srv1979007883=14, srv976100583=28, srv567872626=24, srv321522119=19, srv1426890452=3, srv198474413=15, srv225384920=17, srv904893914=26, srv1230394=0, srv1860857775=13, srv1561509134=6, srv200100457=16, srv1516054427=4, srv1348727905=2} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-28 13:55:20,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-28 13:55:20,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-28 13:55:20,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-28 13:55:20,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-28 13:55:20,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-28 13:55:20,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-28 13:55:20,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-28 13:55:20,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-28 13:55:20,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-28 13:55:20,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-28 13:55:20,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-28 13:55:20,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-28 13:55:20,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-28 13:55:20,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-28 13:55:20,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-28 13:55:20,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-28 13:55:20,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-28 13:55:20,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-28 13:55:20,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-28 13:55:20,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-28 13:55:20,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-28 13:55:20,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-28 13:55:20,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-28 13:55:20,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-28 13:55:20,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-28 13:55:20,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-28 13:55:20,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-28 13:55:20,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-28 13:55:20,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-28 13:55:20,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-28 13:55:20,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-11-28 13:55:20,221 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-28 13:55:20,221 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1230394:90 , srv1271048017:90 , srv1348727905:90 , srv1426890452:90 , srv1516054427:90 , srv1558536274:90 , srv1561509134:90 , srv1572421028:90 , srv1627978022:90 , srv1630282758:90 , srv1787123122:90 , srv182346811:90 , srv1835584236:90 , srv1860857775:90 , srv1979007883:90 , srv198474413:90 , srv200100457:90 , srv225384920:90 , srv263650571:90 , srv321522119:90 , srv352722793:90 , srv43912513:90 , srv472997169:90 , srv565982095:90 , srv567872626:90 , srv852666253:90 , srv904893914:90 , srv926021086:90 , srv976100583:90 , srv981069433:90 }
2023-11-28 13:55:20,236 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=193 (was 361), ProcessCount=170 (was 170), AvailableMemoryMB=6094 (was 6710)