2024-02-07 05:53:11,967 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2024-02-07 05:53:12,280 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2024-02-07 05:53:12,397 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2024-02-07 05:53:12,463 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-02-07 05:53:12,470 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-02-07 05:53:12,484 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=287, ProcessCount=170, AvailableMemoryMB=3758
2024-02-07 05:53:12,486 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-02-07 05:53:12,487 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-02-07 05:53:12,632 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1013625093:84 , srv1015440899:84 , srv1033275732:84 , srv1047781612:84 , srv1056001886:84 , srv1081145974:84 , srv1108031848:84 , srv1131503234:84 , srv1156897424:84 , srv1173579706:84 , srv1205847744:84 , srv1218540672:84 , srv1349155081:84 , srv1353559751:84 , srv1362314971:84 , srv1369994824:84 , srv1473479510:84 , srv1512290147:84 , srv155729234:84 , srv1563780982:84 , srv1567338700:84 , srv1567353743:84 , srv1601380627:84 , srv1659501285:84 , srv1675738140:84 , srv1685387039:84 , srv1704081865:84 , srv1729601309:84 , srv1739685364:84 , srv1770447355:84 , srv1777327737:84 , srv178469099:84 , srv179002558:84 , srv1800062110:84 , srv1800546308:84 , srv1805282164:84 , srv1806672931:84 , srv1812825703:84 , srv183575563:84 , srv1855984034:84 , srv186940237:84 , srv1889736475:84 , srv1913237113:84 , srv191748341:84 , srv1927684111:84 , srv1927835269:84 , srv1931787168:84 , srv1936384502:84 , srv1945394241:84 , srv1949473994:84 , srv1959795903:84 , srv1963374580:84 , srv198877760:84 , srv2000597673:84 , srv2029381852:84 , srv2045428531:84 , srv207065025:84 , srv2076497892:84 , srv2086906375:84 , srv2090655566:84 , srv2096420526:84 , srv210385849:84 , srv2116928068:84 , srv2127988608:84 , srv2135370364:84 , srv222367800:84 , srv251162641:84 , srv270256144:84 , srv310750239:84 , srv401625832:84 , srv480058516:84 , srv495180441:84 , srv505908597:84 , srv517924399:84 , srv523655860:84 , srv54012609:84 , srv57346228:84 , srv577829608:84 , srv594374948:84 , srv610854132:84 , srv685558932:84 , srv685574784:84 , srv691498421:84 , srv692265393:84 , srv704835421:84 , srv737478099:84 , srv751368581:84 , srv779760622:84 , srv782799244:84 , srv800760220:84 , srv824820675:84 , srv832776559:84 , srv852324729:84 , srv880372530:84 , srv910518693:84 , srv916543851:84 , srv92502763:84 , srv940708009:84 , srv971132580:84 , srv1028979127:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2024-02-07 05:53:12,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-07 05:53:12,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1473479510=17, srv852324729=93, srv910518693=95, srv1770447355=30, srv1601380627=23, srv178469099=32, srv210385849=62, srv1659501285=24, srv1218540672=12, srv704835421=85, srv1156897424=9, srv1931787168=47, srv1108031848=7, srv1173579706=10, srv1889736475=42, srv251162641=67, srv1959795903=51, srv1800546308=35, srv577829608=78, srv54012609=76, srv222367800=66, srv1567353743=22, srv1812825703=38, srv179002558=33, srv2045428531=56, srv685558932=81, srv782799244=89, srv1353559751=14, srv880372530=94, srv1945394241=49, srv1855984034=40, srv505908597=73, srv1675738140=25, srv2029381852=55, srv779760622=88, srv517924399=74, srv691498421=83, srv92502763=97, srv1927684111=45, srv1739685364=29, srv1567338700=21, srv1806672931=37, srv1685387039=26, srv1047781612=4, srv207065025=57, srv310750239=69, srv1563780982=20, srv1056001886=5, srv800760220=90, srv2086906375=59, srv832776559=92, srv1729601309=28, srv2096420526=61, srv1033275732=3, srv594374948=79, srv1013625093=0, srv1028979127=2, srv1369994824=16, srv1081145974=6, srv2127988608=64, srv692265393=84, srv1015440899=1, srv183575563=39, srv495180441=72, srv1949473994=50, srv1349155081=13, srv971132580=99, srv751368581=87, srv2076497892=58, srv916543851=96, srv2116928068=63, srv2000597673=54, srv1704081865=27, srv186940237=41, srv480058516=71, srv1777327737=31, srv2135370364=65, srv940708009=98, srv1800062110=34, srv1963374580=52, srv1362314971=15, srv270256144=68, srv1205847744=11, srv610854132=80, srv523655860=75, srv685574784=82, srv155729234=19, srv191748341=44, srv198877760=53, srv824820675=91, srv1913237113=43, srv1936384502=48, srv1927835269=46, srv1805282164=36, srv2090655566=60, srv401625832=70, srv57346228=77, srv1131503234=8, srv737478099=86, srv1512290147=18} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-07 05:53:12,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-07 05:53:12,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-07 05:53:12,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-07 05:53:12,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-07 05:53:12,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-07 05:53:12,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-07 05:53:12,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-07 05:53:12,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-07 05:53:12,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-07 05:53:12,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-07 05:53:12,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-07 05:53:12,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-07 05:53:12,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-07 05:53:12,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-07 05:53:12,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-07 05:53:12,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-07 05:53:12,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-07 05:53:12,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-07 05:53:12,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-07 05:53:12,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-07 05:53:12,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-07 05:53:12,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-07 05:53:12,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-07 05:53:12,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-07 05:53:12,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-07 05:53:12,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-07 05:53:12,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-07 05:53:12,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-07 05:53:12,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-07 05:53:12,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-07 05:53:12,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-02-07 05:53:12,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-02-07 05:53:12,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-02-07 05:53:12,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-02-07 05:53:12,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-02-07 05:53:12,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-02-07 05:53:12,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-02-07 05:53:12,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-02-07 05:53:12,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-02-07 05:53:12,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-02-07 05:53:12,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-02-07 05:53:12,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-02-07 05:53:12,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-02-07 05:53:12,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-02-07 05:53:12,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-02-07 05:53:12,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-02-07 05:53:12,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-02-07 05:53:12,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-02-07 05:53:12,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-02-07 05:53:12,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-02-07 05:53:12,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-02-07 05:53:12,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-02-07 05:53:12,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-02-07 05:53:12,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-02-07 05:53:12,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-02-07 05:53:12,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-02-07 05:53:12,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-02-07 05:53:12,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-02-07 05:53:12,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-02-07 05:53:12,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-02-07 05:53:12,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-02-07 05:53:12,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-02-07 05:53:12,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-02-07 05:53:12,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-02-07 05:53:12,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-02-07 05:53:12,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-02-07 05:53:12,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-02-07 05:53:12,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-02-07 05:53:12,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-02-07 05:53:12,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-02-07 05:53:12,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-02-07 05:53:12,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-02-07 05:53:12,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-02-07 05:53:12,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-02-07 05:53:12,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-02-07 05:53:12,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-02-07 05:53:12,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-02-07 05:53:12,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-02-07 05:53:12,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-02-07 05:53:12,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-02-07 05:53:12,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-02-07 05:53:12,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-02-07 05:53:12,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-02-07 05:53:12,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-02-07 05:53:12,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-02-07 05:53:12,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-02-07 05:53:12,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-02-07 05:53:12,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-02-07 05:53:12,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-02-07 05:53:12,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-02-07 05:53:12,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-02-07 05:53:12,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-02-07 05:53:12,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-02-07 05:53:12,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-02-07 05:53:12,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-02-07 05:53:12,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-02-07 05:53:12,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-02-07 05:53:12,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-02-07 05:53:12,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-02-07 05:53:12,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-02-07 05:53:12,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-02-07 05:53:12,939 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-07 05:53:12,939 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-02-07 05:53:12,940 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-02-07 05:53:13,310 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 531 ms to try 5000 different iterations.  Found a solution that moves 4720 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17657943049679176. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039143466873511154, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05693440414072767, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5244444444444445, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039143466873511154, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17116666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24441666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-07 05:53:13,328 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1601380627:82 , srv1013625093:83 , srv1205847744:83 , srv210385849:83 , srv310750239:83 , srv523655860:83 , srv751368581:83 , srv779760622:83 , srv1081145974:84 , srv1131503234:84 , srv1567353743:84 , srv1659501285:84 , srv179002558:84 , srv1889736475:84 , srv1931787168:84 , srv517924399:84 , srv782799244:84 , srv832776559:84 , srv1362314971:85 , srv155729234:85 , srv1563780982:85 , srv1675738140:85 , srv1805282164:85 , srv1806672931:85 , srv191748341:85 , srv1927684111:85 , srv207065025:85 , srv2086906375:85 , srv2127988608:85 , srv2135370364:85 , srv401625832:85 , srv495180441:85 , srv852324729:85 , srv916543851:85 , srv92502763:85 , srv971132580:85 , srv1047781612:86 , srv1353559751:86 , srv1369994824:86 , srv1685387039:86 , srv1704081865:86 , srv183575563:86 , srv1855984034:86 , srv186940237:86 , srv1913237113:86 , srv1949473994:86 , srv2029381852:86 , srv2090655566:86 , srv2116928068:86 , srv222367800:86 , srv480058516:86 , srv505908597:86 , srv57346228:86 , srv685558932:86 , srv685574784:86 , srv691498421:86 , srv692265393:86 , srv704835421:86 , srv1218540672:87 , srv1349155081:87 , srv1567338700:87 , srv178469099:87 , srv1800546308:87 , srv1927835269:87 , srv1959795903:87 , srv198877760:87 , srv2076497892:87 , srv594374948:87 , srv824820675:87 , srv880372530:87 , srv910518693:87 , srv1033275732:88 , srv1056001886:88 , srv1108031848:88 , srv1156897424:88 , srv2045428531:88 , srv2096420526:88 , srv251162641:88 , srv270256144:88 , srv54012609:88 , srv577829608:88 , srv610854132:88 , srv1173579706:89 , srv1729601309:89 , srv1945394241:89 , srv1963374580:89 , srv1015440899:90 , srv1512290147:90 , srv1739685364:90 , srv1770447355:90 , srv1800062110:90 , srv2000597673:90 , srv800760220:90 , srv1777327737:91 , srv737478099:91 , srv1473479510:92 , srv1936384502:92 , srv940708009:92 , srv1812825703:94 , srv1028979127:438 }
2024-02-07 05:53:13,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-07 05:53:13,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1473479510=17, srv852324729=93, srv910518693=95, srv1770447355=30, srv1601380627=23, srv178469099=32, srv210385849=62, srv1659501285=24, srv1218540672=12, srv704835421=85, srv1156897424=9, srv1931787168=47, srv1108031848=7, srv1173579706=10, srv1889736475=42, srv251162641=67, srv1959795903=51, srv1800546308=35, srv577829608=78, srv54012609=76, srv222367800=66, srv1567353743=22, srv1812825703=38, srv179002558=33, srv2045428531=56, srv685558932=81, srv782799244=89, srv1353559751=14, srv880372530=94, srv1945394241=49, srv1855984034=40, srv505908597=73, srv1675738140=25, srv2029381852=55, srv779760622=88, srv517924399=74, srv691498421=83, srv92502763=97, srv1927684111=45, srv1739685364=29, srv1567338700=21, srv1806672931=37, srv1685387039=26, srv1047781612=4, srv207065025=57, srv310750239=69, srv1563780982=20, srv1056001886=5, srv800760220=90, srv2086906375=59, srv832776559=92, srv1729601309=28, srv2096420526=61, srv1033275732=3, srv594374948=79, srv1013625093=0, srv1028979127=2, srv1369994824=16, srv1081145974=6, srv2127988608=64, srv692265393=84, srv1015440899=1, srv183575563=39, srv495180441=72, srv1949473994=50, srv1349155081=13, srv971132580=99, srv751368581=87, srv2076497892=58, srv916543851=96, srv2116928068=63, srv2000597673=54, srv1704081865=27, srv186940237=41, srv480058516=71, srv1777327737=31, srv2135370364=65, srv940708009=98, srv1800062110=34, srv1963374580=52, srv1362314971=15, srv270256144=68, srv1205847744=11, srv610854132=80, srv523655860=75, srv685574784=82, srv155729234=19, srv191748341=44, srv198877760=53, srv824820675=91, srv1913237113=43, srv1936384502=48, srv1927835269=46, srv1805282164=36, srv2090655566=60, srv401625832=70, srv57346228=77, srv1131503234=8, srv737478099=86, srv1512290147=18} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-07 05:53:13,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-07 05:53:13,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-07 05:53:13,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-07 05:53:13,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-07 05:53:13,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-07 05:53:13,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-07 05:53:13,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-07 05:53:13,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-07 05:53:13,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-07 05:53:13,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-07 05:53:13,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-07 05:53:13,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-07 05:53:13,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-07 05:53:13,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-07 05:53:13,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-07 05:53:13,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-07 05:53:13,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-07 05:53:13,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-07 05:53:13,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-07 05:53:13,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-07 05:53:13,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-07 05:53:13,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-07 05:53:13,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-07 05:53:13,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-07 05:53:13,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-07 05:53:13,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-07 05:53:13,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-07 05:53:13,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-07 05:53:13,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-07 05:53:13,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-07 05:53:13,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-02-07 05:53:13,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-02-07 05:53:13,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-02-07 05:53:13,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-02-07 05:53:13,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-02-07 05:53:13,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-02-07 05:53:13,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-02-07 05:53:13,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-02-07 05:53:13,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-02-07 05:53:13,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-02-07 05:53:13,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-02-07 05:53:13,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-02-07 05:53:13,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-02-07 05:53:13,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-02-07 05:53:13,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-02-07 05:53:13,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-02-07 05:53:13,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-02-07 05:53:13,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-02-07 05:53:13,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-02-07 05:53:13,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-02-07 05:53:13,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-02-07 05:53:13,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-02-07 05:53:13,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-02-07 05:53:13,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-02-07 05:53:13,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-02-07 05:53:13,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-02-07 05:53:13,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-02-07 05:53:13,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-02-07 05:53:13,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-02-07 05:53:13,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-02-07 05:53:13,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-02-07 05:53:13,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-02-07 05:53:13,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-02-07 05:53:13,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-02-07 05:53:13,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-02-07 05:53:13,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-02-07 05:53:13,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-02-07 05:53:13,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-02-07 05:53:13,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-02-07 05:53:13,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-02-07 05:53:13,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-02-07 05:53:13,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-02-07 05:53:13,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-02-07 05:53:13,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-02-07 05:53:13,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-02-07 05:53:13,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-02-07 05:53:13,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-02-07 05:53:13,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-02-07 05:53:13,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-02-07 05:53:13,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-02-07 05:53:13,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-02-07 05:53:13,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-02-07 05:53:13,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-02-07 05:53:13,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-02-07 05:53:13,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-02-07 05:53:13,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-02-07 05:53:13,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-02-07 05:53:13,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-02-07 05:53:13,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-02-07 05:53:13,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-02-07 05:53:13,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-02-07 05:53:13,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-02-07 05:53:13,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-02-07 05:53:13,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-02-07 05:53:13,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-02-07 05:53:13,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-02-07 05:53:13,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-02-07 05:53:13,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-02-07 05:53:13,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-02-07 05:53:13,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-02-07 05:53:13,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-02-07 05:53:13,537 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039143466873511154, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05693440414072767, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039143466873511154, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17116666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24441666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-07 05:53:13,537 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-02-07 05:53:13,538 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17654638035039683, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039143466873511154, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05693440414072767, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039143466873511154, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17116666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24441666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-02-07 05:53:13,859 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 465 ms to try 5000 different iterations.  Found a solution that moves 2627 regions; Going from a computed imbalance of 0.17654638035039683 to a new imbalance of 0.014234034171735986. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02084928308270995); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03408578337822522, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29188888888888886, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02084928308270995); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04591666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-07 05:53:13,873 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv940708009:78 , srv1812825703:79 , srv1473479510:80 , srv1512290147:80 , srv737478099:80 , srv1739685364:81 , srv1936384502:81 , srv1945394241:81 , srv1056001886:82 , srv1567338700:82 , srv1777327737:82 , srv1963374580:82 , srv2096420526:82 , srv800760220:82 , srv1353559751:83 , srv1770447355:83 , srv1800062110:83 , srv1927835269:83 , srv2000597673:83 , srv610854132:83 , srv1015440899:84 , srv1156897424:84 , srv1349155081:84 , srv2045428531:84 , srv505908597:84 , srv594374948:84 , srv1033275732:85 , srv1047781612:85 , srv1173579706:85 , srv1218540672:85 , srv1369994824:85 , srv1729601309:85 , srv183575563:85 , srv1913237113:85 , srv1949473994:85 , srv2029381852:85 , srv207065025:85 , srv2076497892:85 , srv222367800:85 , srv270256144:85 , srv57346228:85 , srv685574784:85 , srv910518693:85 , srv155729234:86 , srv178469099:86 , srv1800546308:86 , srv1889736475:86 , srv1959795903:86 , srv198877760:86 , srv2086906375:86 , srv2116928068:86 , srv251162641:86 , srv310750239:86 , srv480058516:86 , srv577829608:86 , srv691498421:86 , srv782799244:86 , srv824820675:86 , srv852324729:86 , srv880372530:86 , srv1362314971:87 , srv1563780982:87 , srv1704081865:87 , srv179002558:87 , srv1805282164:87 , srv1855984034:87 , srv191748341:87 , srv1931787168:87 , srv401625832:87 , srv495180441:87 , srv685558932:87 , srv692265393:87 , srv704835421:87 , srv751368581:87 , srv832776559:87 , srv916543851:87 , srv92502763:87 , srv1013625093:88 , srv1108031848:88 , srv1567353743:88 , srv1659501285:88 , srv1685387039:88 , srv1806672931:88 , srv186940237:88 , srv1927684111:88 , srv2090655566:88 , srv2127988608:88 , srv2135370364:88 , srv517924399:88 , srv1131503234:89 , srv1675738140:89 , srv971132580:89 , srv1081145974:90 , srv1205847744:90 , srv1601380627:90 , srv210385849:90 , srv523655860:90 , srv779760622:92 , srv54012609:94 , srv1028979127:521 }
2024-02-07 05:53:13,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-07 05:53:13,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1473479510=17, srv852324729=93, srv910518693=95, srv1770447355=30, srv1601380627=23, srv178469099=32, srv210385849=62, srv1659501285=24, srv1218540672=12, srv704835421=85, srv1156897424=9, srv1931787168=47, srv1108031848=7, srv1173579706=10, srv1889736475=42, srv251162641=67, srv1959795903=51, srv1800546308=35, srv577829608=78, srv54012609=76, srv222367800=66, srv1567353743=22, srv1812825703=38, srv179002558=33, srv2045428531=56, srv685558932=81, srv782799244=89, srv1353559751=14, srv880372530=94, srv1945394241=49, srv1855984034=40, srv505908597=73, srv1675738140=25, srv2029381852=55, srv779760622=88, srv517924399=74, srv691498421=83, srv92502763=97, srv1927684111=45, srv1739685364=29, srv1567338700=21, srv1806672931=37, srv1685387039=26, srv1047781612=4, srv207065025=57, srv310750239=69, srv1563780982=20, srv1056001886=5, srv800760220=90, srv2086906375=59, srv832776559=92, srv1729601309=28, srv2096420526=61, srv1033275732=3, srv594374948=79, srv1013625093=0, srv1028979127=2, srv1369994824=16, srv1081145974=6, srv2127988608=64, srv692265393=84, srv1015440899=1, srv183575563=39, srv495180441=72, srv1949473994=50, srv1349155081=13, srv971132580=99, srv751368581=87, srv2076497892=58, srv916543851=96, srv2116928068=63, srv2000597673=54, srv1704081865=27, srv186940237=41, srv480058516=71, srv1777327737=31, srv2135370364=65, srv940708009=98, srv1800062110=34, srv1963374580=52, srv1362314971=15, srv270256144=68, srv1205847744=11, srv610854132=80, srv523655860=75, srv685574784=82, srv155729234=19, srv191748341=44, srv198877760=53, srv824820675=91, srv1913237113=43, srv1936384502=48, srv1927835269=46, srv1805282164=36, srv2090655566=60, srv401625832=70, srv57346228=77, srv1131503234=8, srv737478099=86, srv1512290147=18} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-07 05:53:13,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-07 05:53:13,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-07 05:53:13,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-07 05:53:13,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-07 05:53:13,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-07 05:53:13,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-07 05:53:13,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-07 05:53:13,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-07 05:53:13,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-07 05:53:13,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-07 05:53:13,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-07 05:53:13,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-07 05:53:13,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-07 05:53:13,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-07 05:53:13,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-07 05:53:13,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-07 05:53:13,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-07 05:53:13,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-07 05:53:13,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-07 05:53:13,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-07 05:53:13,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-07 05:53:13,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-07 05:53:13,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-07 05:53:13,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-07 05:53:13,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-07 05:53:13,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-07 05:53:13,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-07 05:53:13,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-07 05:53:13,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-07 05:53:13,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-07 05:53:13,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-02-07 05:53:13,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-02-07 05:53:13,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-02-07 05:53:13,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-02-07 05:53:13,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-02-07 05:53:13,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-02-07 05:53:13,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-02-07 05:53:13,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-02-07 05:53:13,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-02-07 05:53:13,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-02-07 05:53:13,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-02-07 05:53:13,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-02-07 05:53:13,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-02-07 05:53:13,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-02-07 05:53:13,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-02-07 05:53:13,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-02-07 05:53:13,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-02-07 05:53:13,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-02-07 05:53:13,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-02-07 05:53:13,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-02-07 05:53:13,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-02-07 05:53:13,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-02-07 05:53:13,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-02-07 05:53:13,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-02-07 05:53:13,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-02-07 05:53:13,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-02-07 05:53:13,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-02-07 05:53:13,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-02-07 05:53:13,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-02-07 05:53:13,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-02-07 05:53:13,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-02-07 05:53:13,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-02-07 05:53:13,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-02-07 05:53:13,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-02-07 05:53:13,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-02-07 05:53:13,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-02-07 05:53:13,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-02-07 05:53:13,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-02-07 05:53:13,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-02-07 05:53:13,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-02-07 05:53:13,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-02-07 05:53:13,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-02-07 05:53:13,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-02-07 05:53:13,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-02-07 05:53:13,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-02-07 05:53:13,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-02-07 05:53:13,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-02-07 05:53:13,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-02-07 05:53:13,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-02-07 05:53:13,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-02-07 05:53:13,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-02-07 05:53:13,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-02-07 05:53:13,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-02-07 05:53:13,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-02-07 05:53:13,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-02-07 05:53:13,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-02-07 05:53:13,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-02-07 05:53:13,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-02-07 05:53:13,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-02-07 05:53:13,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-02-07 05:53:13,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-02-07 05:53:13,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-02-07 05:53:13,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-02-07 05:53:13,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-02-07 05:53:13,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-02-07 05:53:13,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-02-07 05:53:13,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-02-07 05:53:13,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-02-07 05:53:13,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-02-07 05:53:13,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-02-07 05:53:13,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-02-07 05:53:14,072 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02084928308270995); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03408578337822522, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02084928308270995); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04591666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-07 05:53:14,072 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-02-07 05:53:14,072 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.01421563952457931, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02084928308270995); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03408578337822522, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02084928308270995); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04591666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-02-07 05:53:14,317 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 381 ms to try 5000 different iterations.  Found a solution that moves 969 regions; Going from a computed imbalance of 0.01421563952457931 to a new imbalance of 4.7483169888736833E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008701733425533873); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.01300090646930317); MoveCostFunction : (multiplier=7.0, imbalance=0.10766666666666666, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.008701733425533873); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00325); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-07 05:53:14,322 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv54012609:77 , srv1108031848:80 , srv1729601309:82 , srv2090655566:82 , srv779760622:82 , srv971132580:82 , srv1081145974:83 , srv1173579706:83 , srv1205847744:83 , srv1704081865:83 , srv1770447355:83 , srv1777327737:83 , srv1812825703:83 , srv1936384502:83 , srv198877760:83 , srv210385849:83 , srv480058516:83 , srv1015440899:84 , srv1131503234:84 , srv1675738140:84 , srv1800062110:84 , srv1800546308:84 , srv1806672931:84 , srv186940237:84 , srv1927684111:84 , srv1959795903:84 , srv2000597673:84 , srv2135370364:84 , srv251162641:84 , srv270256144:84 , srv401625832:84 , srv523655860:84 , srv577829608:84 , srv691498421:84 , srv692265393:84 , srv800760220:84 , srv880372530:84 , srv910518693:84 , srv1033275732:85 , srv1563780982:85 , srv1567353743:85 , srv1659501285:85 , srv1685387039:85 , srv178469099:85 , srv183575563:85 , srv1855984034:85 , srv191748341:85 , srv2029381852:85 , srv2076497892:85 , srv2116928068:85 , srv2127988608:85 , srv685558932:85 , srv704835421:85 , srv824820675:85 , srv1013625093:86 , srv1047781612:86 , srv1156897424:86 , srv1218540672:86 , srv1349155081:86 , srv1362314971:86 , srv1473479510:86 , srv1512290147:86 , srv155729234:86 , srv1601380627:86 , srv1739685364:86 , srv179002558:86 , srv1913237113:86 , srv2045428531:86 , srv207065025:86 , srv2086906375:86 , srv222367800:86 , srv495180441:86 , srv517924399:86 , srv610854132:86 , srv685574784:86 , srv737478099:86 , srv751368581:86 , srv1369994824:87 , srv1805282164:87 , srv1889736475:87 , srv1927835269:87 , srv1931787168:87 , srv1949473994:87 , srv1963374580:87 , srv2096420526:87 , srv310750239:87 , srv594374948:87 , srv832776559:87 , srv852324729:87 , srv92502763:87 , srv1056001886:88 , srv1353559751:88 , srv1945394241:88 , srv505908597:88 , srv57346228:88 , srv782799244:88 , srv916543851:88 , srv940708009:88 , srv1567338700:89 , srv1028979127:576 }
2024-02-07 05:53:14,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-07 05:53:14,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1473479510=17, srv852324729=93, srv910518693=95, srv1770447355=30, srv1601380627=23, srv178469099=32, srv210385849=62, srv1659501285=24, srv1218540672=12, srv704835421=85, srv1156897424=9, srv1931787168=47, srv1108031848=7, srv1173579706=10, srv1889736475=42, srv251162641=67, srv1959795903=51, srv1800546308=35, srv577829608=78, srv54012609=76, srv222367800=66, srv1567353743=22, srv1812825703=38, srv179002558=33, srv2045428531=56, srv685558932=81, srv782799244=89, srv1353559751=14, srv880372530=94, srv1945394241=49, srv1855984034=40, srv505908597=73, srv1675738140=25, srv2029381852=55, srv779760622=88, srv517924399=74, srv691498421=83, srv92502763=97, srv1927684111=45, srv1739685364=29, srv1567338700=21, srv1806672931=37, srv1685387039=26, srv1047781612=4, srv207065025=57, srv310750239=69, srv1563780982=20, srv1056001886=5, srv800760220=90, srv2086906375=59, srv832776559=92, srv1729601309=28, srv2096420526=61, srv1033275732=3, srv594374948=79, srv1013625093=0, srv1028979127=2, srv1369994824=16, srv1081145974=6, srv2127988608=64, srv692265393=84, srv1015440899=1, srv183575563=39, srv495180441=72, srv1949473994=50, srv1349155081=13, srv971132580=99, srv751368581=87, srv2076497892=58, srv916543851=96, srv2116928068=63, srv2000597673=54, srv1704081865=27, srv186940237=41, srv480058516=71, srv1777327737=31, srv2135370364=65, srv940708009=98, srv1800062110=34, srv1963374580=52, srv1362314971=15, srv270256144=68, srv1205847744=11, srv610854132=80, srv523655860=75, srv685574784=82, srv155729234=19, srv191748341=44, srv198877760=53, srv824820675=91, srv1913237113=43, srv1936384502=48, srv1927835269=46, srv1805282164=36, srv2090655566=60, srv401625832=70, srv57346228=77, srv1131503234=8, srv737478099=86, srv1512290147=18} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-07 05:53:14,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-07 05:53:14,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-07 05:53:14,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-07 05:53:14,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-07 05:53:14,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-07 05:53:14,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-07 05:53:14,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-07 05:53:14,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-07 05:53:14,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-07 05:53:14,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-07 05:53:14,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-07 05:53:14,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-07 05:53:14,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-07 05:53:14,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-07 05:53:14,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-07 05:53:14,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-07 05:53:14,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-07 05:53:14,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-07 05:53:14,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-07 05:53:14,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-07 05:53:14,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-07 05:53:14,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-07 05:53:14,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-07 05:53:14,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-07 05:53:14,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-07 05:53:14,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-07 05:53:14,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-07 05:53:14,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-07 05:53:14,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-07 05:53:14,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-07 05:53:14,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-02-07 05:53:14,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-02-07 05:53:14,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-02-07 05:53:14,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-02-07 05:53:14,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-02-07 05:53:14,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-02-07 05:53:14,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-02-07 05:53:14,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-02-07 05:53:14,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-02-07 05:53:14,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-02-07 05:53:14,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-02-07 05:53:14,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-02-07 05:53:14,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-02-07 05:53:14,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-02-07 05:53:14,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-02-07 05:53:14,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-02-07 05:53:14,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-02-07 05:53:14,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-02-07 05:53:14,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-02-07 05:53:14,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-02-07 05:53:14,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-02-07 05:53:14,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-02-07 05:53:14,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-02-07 05:53:14,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-02-07 05:53:14,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-02-07 05:53:14,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-02-07 05:53:14,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-02-07 05:53:14,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-02-07 05:53:14,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-02-07 05:53:14,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-02-07 05:53:14,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-02-07 05:53:14,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-02-07 05:53:14,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-02-07 05:53:14,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-02-07 05:53:14,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-02-07 05:53:14,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-02-07 05:53:14,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-02-07 05:53:14,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-02-07 05:53:14,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-02-07 05:53:14,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-02-07 05:53:14,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-02-07 05:53:14,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-02-07 05:53:14,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-02-07 05:53:14,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-02-07 05:53:14,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-02-07 05:53:14,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-02-07 05:53:14,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-02-07 05:53:14,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-02-07 05:53:14,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-02-07 05:53:14,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-02-07 05:53:14,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-02-07 05:53:14,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-02-07 05:53:14,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-02-07 05:53:14,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-02-07 05:53:14,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-02-07 05:53:14,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-02-07 05:53:14,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-02-07 05:53:14,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-02-07 05:53:14,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-02-07 05:53:14,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-02-07 05:53:14,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-02-07 05:53:14,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-02-07 05:53:14,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-02-07 05:53:14,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-02-07 05:53:14,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-02-07 05:53:14,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-02-07 05:53:14,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-02-07 05:53:14,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-02-07 05:53:14,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-02-07 05:53:14,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-02-07 05:53:14,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-02-07 05:53:14,458 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008701733425533873); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.01300090646930317); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.008701733425533873); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00325); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-07 05:53:14,458 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-02-07 05:53:14,459 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=4.68046615866881E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008701733425533873); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.01300090646930317); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.008701733425533873); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00325); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-02-07 05:53:14,892 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 533 ms to try 5000 different iterations.  Found a solution that moves 256 regions; Going from a computed imbalance of 4.68046615866881E-4 to a new imbalance of 1.2867688775374398E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0019534423844256277); MoveCostFunction : (multiplier=7.0, imbalance=0.028444444444444446, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-07 05:53:14,893 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv916543851:82 , srv1685387039:83 , srv57346228:83 , srv92502763:83 , srv1015440899:84 , srv1033275732:84 , srv1156897424:84 , srv1173579706:84 , srv1218540672:84 , srv1353559751:84 , srv1362314971:84 , srv1369994824:84 , srv1473479510:84 , srv1567338700:84 , srv1567353743:84 , srv1601380627:84 , srv1675738140:84 , srv1805282164:84 , srv186940237:84 , srv1931787168:84 , srv1945394241:84 , srv1949473994:84 , srv1959795903:84 , srv1963374580:84 , srv2045428531:84 , srv2127988608:84 , srv251162641:84 , srv495180441:84 , srv505908597:84 , srv517924399:84 , srv54012609:84 , srv577829608:84 , srv594374948:84 , srv704835421:84 , srv782799244:84 , srv824820675:84 , srv832776559:84 , srv852324729:84 , srv940708009:84 , srv1013625093:85 , srv1047781612:85 , srv1056001886:85 , srv1081145974:85 , srv1349155081:85 , srv155729234:85 , srv1563780982:85 , srv1659501285:85 , srv1739685364:85 , srv1770447355:85 , srv178469099:85 , srv179002558:85 , srv1800062110:85 , srv1800546308:85 , srv1806672931:85 , srv1812825703:85 , srv1855984034:85 , srv1889736475:85 , srv1913237113:85 , srv191748341:85 , srv1927684111:85 , srv1927835269:85 , srv2000597673:85 , srv2076497892:85 , srv2086906375:85 , srv2116928068:85 , srv2135370364:85 , srv222367800:85 , srv270256144:85 , srv401625832:85 , srv523655860:85 , srv610854132:85 , srv685558932:85 , srv685574784:85 , srv692265393:85 , srv737478099:85 , srv779760622:85 , srv800760220:85 , srv880372530:85 , srv971132580:85 , srv1108031848:86 , srv1131503234:86 , srv1205847744:86 , srv1512290147:86 , srv1704081865:86 , srv1729601309:86 , srv183575563:86 , srv1936384502:86 , srv198877760:86 , srv2029381852:86 , srv207065025:86 , srv2090655566:86 , srv2096420526:86 , srv210385849:86 , srv310750239:86 , srv691498421:86 , srv751368581:86 , srv910518693:86 , srv1777327737:87 , srv480058516:87 , srv1028979127:607 }
2024-02-07 05:53:14,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-07 05:53:14,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1473479510=17, srv852324729=93, srv910518693=95, srv1770447355=30, srv1601380627=23, srv178469099=32, srv210385849=62, srv1659501285=24, srv1218540672=12, srv704835421=85, srv1156897424=9, srv1931787168=47, srv1108031848=7, srv1173579706=10, srv1889736475=42, srv251162641=67, srv1959795903=51, srv1800546308=35, srv577829608=78, srv54012609=76, srv222367800=66, srv1567353743=22, srv1812825703=38, srv179002558=33, srv2045428531=56, srv685558932=81, srv782799244=89, srv1353559751=14, srv880372530=94, srv1945394241=49, srv1855984034=40, srv505908597=73, srv1675738140=25, srv2029381852=55, srv779760622=88, srv517924399=74, srv691498421=83, srv92502763=97, srv1927684111=45, srv1739685364=29, srv1567338700=21, srv1806672931=37, srv1685387039=26, srv1047781612=4, srv207065025=57, srv310750239=69, srv1563780982=20, srv1056001886=5, srv800760220=90, srv2086906375=59, srv832776559=92, srv1729601309=28, srv2096420526=61, srv1033275732=3, srv594374948=79, srv1013625093=0, srv1028979127=2, srv1369994824=16, srv1081145974=6, srv2127988608=64, srv692265393=84, srv1015440899=1, srv183575563=39, srv495180441=72, srv1949473994=50, srv1349155081=13, srv971132580=99, srv751368581=87, srv2076497892=58, srv916543851=96, srv2116928068=63, srv2000597673=54, srv1704081865=27, srv186940237=41, srv480058516=71, srv1777327737=31, srv2135370364=65, srv940708009=98, srv1800062110=34, srv1963374580=52, srv1362314971=15, srv270256144=68, srv1205847744=11, srv610854132=80, srv523655860=75, srv685574784=82, srv155729234=19, srv191748341=44, srv198877760=53, srv824820675=91, srv1913237113=43, srv1936384502=48, srv1927835269=46, srv1805282164=36, srv2090655566=60, srv401625832=70, srv57346228=77, srv1131503234=8, srv737478099=86, srv1512290147=18} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-07 05:53:14,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-07 05:53:14,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-07 05:53:14,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-07 05:53:14,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-07 05:53:14,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-07 05:53:14,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-07 05:53:14,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-07 05:53:14,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-07 05:53:14,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-07 05:53:14,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-07 05:53:14,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-07 05:53:14,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-07 05:53:14,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-07 05:53:14,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-07 05:53:14,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-07 05:53:14,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-07 05:53:14,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-07 05:53:14,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-07 05:53:14,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-07 05:53:14,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-07 05:53:14,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-07 05:53:14,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-07 05:53:14,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-07 05:53:14,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-07 05:53:14,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-07 05:53:14,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-07 05:53:14,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-07 05:53:14,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-07 05:53:14,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-07 05:53:14,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-07 05:53:14,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-02-07 05:53:14,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-02-07 05:53:14,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-02-07 05:53:14,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-02-07 05:53:14,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-02-07 05:53:14,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-02-07 05:53:14,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-02-07 05:53:14,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-02-07 05:53:14,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-02-07 05:53:14,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-02-07 05:53:14,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-02-07 05:53:14,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-02-07 05:53:14,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-02-07 05:53:14,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-02-07 05:53:14,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-02-07 05:53:14,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-02-07 05:53:14,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-02-07 05:53:14,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-02-07 05:53:14,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-02-07 05:53:14,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-02-07 05:53:14,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-02-07 05:53:14,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-02-07 05:53:14,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-02-07 05:53:14,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-02-07 05:53:14,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-02-07 05:53:14,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-02-07 05:53:14,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-02-07 05:53:14,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-02-07 05:53:14,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-02-07 05:53:14,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-02-07 05:53:14,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-02-07 05:53:14,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-02-07 05:53:14,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-02-07 05:53:14,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-02-07 05:53:14,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-02-07 05:53:14,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-02-07 05:53:14,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-02-07 05:53:14,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-02-07 05:53:14,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-02-07 05:53:14,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-02-07 05:53:14,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-02-07 05:53:14,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-02-07 05:53:14,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-02-07 05:53:14,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-02-07 05:53:14,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-02-07 05:53:14,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-02-07 05:53:14,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-02-07 05:53:14,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-02-07 05:53:14,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-02-07 05:53:14,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-02-07 05:53:14,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-02-07 05:53:14,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-02-07 05:53:14,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-02-07 05:53:14,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-02-07 05:53:14,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-02-07 05:53:14,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-02-07 05:53:14,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-02-07 05:53:14,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-02-07 05:53:14,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-02-07 05:53:14,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-02-07 05:53:14,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-02-07 05:53:14,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-02-07 05:53:14,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-02-07 05:53:14,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-02-07 05:53:14,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-02-07 05:53:14,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-02-07 05:53:14,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-02-07 05:53:14,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-02-07 05:53:14,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-02-07 05:53:14,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-02-07 05:53:14,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-02-07 05:53:15,052 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.1075138462428321E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0019534423844256277); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-07 05:53:15,052 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv916543851:82 , srv1685387039:83 , srv57346228:83 , srv92502763:83 , srv1015440899:84 , srv1033275732:84 , srv1156897424:84 , srv1173579706:84 , srv1218540672:84 , srv1353559751:84 , srv1362314971:84 , srv1369994824:84 , srv1473479510:84 , srv1567338700:84 , srv1567353743:84 , srv1601380627:84 , srv1675738140:84 , srv1805282164:84 , srv186940237:84 , srv1931787168:84 , srv1945394241:84 , srv1949473994:84 , srv1959795903:84 , srv1963374580:84 , srv2045428531:84 , srv2127988608:84 , srv251162641:84 , srv495180441:84 , srv505908597:84 , srv517924399:84 , srv54012609:84 , srv577829608:84 , srv594374948:84 , srv704835421:84 , srv782799244:84 , srv824820675:84 , srv832776559:84 , srv852324729:84 , srv940708009:84 , srv1013625093:85 , srv1047781612:85 , srv1056001886:85 , srv1081145974:85 , srv1349155081:85 , srv155729234:85 , srv1563780982:85 , srv1659501285:85 , srv1739685364:85 , srv1770447355:85 , srv178469099:85 , srv179002558:85 , srv1800062110:85 , srv1800546308:85 , srv1806672931:85 , srv1812825703:85 , srv1855984034:85 , srv1889736475:85 , srv1913237113:85 , srv191748341:85 , srv1927684111:85 , srv1927835269:85 , srv2000597673:85 , srv2076497892:85 , srv2086906375:85 , srv2116928068:85 , srv2135370364:85 , srv222367800:85 , srv270256144:85 , srv401625832:85 , srv523655860:85 , srv610854132:85 , srv685558932:85 , srv685574784:85 , srv692265393:85 , srv737478099:85 , srv779760622:85 , srv800760220:85 , srv880372530:85 , srv971132580:85 , srv1108031848:86 , srv1131503234:86 , srv1205847744:86 , srv1512290147:86 , srv1704081865:86 , srv1729601309:86 , srv183575563:86 , srv1936384502:86 , srv198877760:86 , srv2029381852:86 , srv207065025:86 , srv2090655566:86 , srv2096420526:86 , srv210385849:86 , srv310750239:86 , srv691498421:86 , srv751368581:86 , srv910518693:86 , srv1777327737:87 , srv480058516:87 , srv1028979127:607 }
2024-02-07 05:53:15,084 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=313 (was 287) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=2922 (was 3758)
2024-02-07 05:53:15,122 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=313, ProcessCount=170, AvailableMemoryMB=2887
2024-02-07 05:53:15,123 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-02-07 05:53:15,123 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2024-02-07 05:53:15,139 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1080081498:84 , srv1213557031:84 , srv1248539515:84 , srv128607315:84 , srv1296799755:84 , srv1309778401:84 , srv1336926690:84 , srv1404382841:84 , srv1439296539:84 , srv1550101630:84 , srv1553762281:84 , srv1572080445:84 , srv1663181123:84 , srv2003315676:84 , srv2011181248:84 , srv2012765552:84 , srv336359066:84 , srv410088987:84 , srv494976957:84 , srv530009806:84 , srv548930996:84 , srv590471835:84 , srv624508045:84 , srv691736341:84 , srv714730745:84 , srv757339386:84 , srv934295612:84 , srv958252600:84 , srv986543697:84 , srv1540371330:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2024-02-07 05:53:15,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-07 05:53:15,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1404382841=7, srv757339386=26, srv410088987=18, srv958252600=28, srv128607315=3, srv494976957=19, srv1572080445=12, srv1296799755=4, srv548930996=21, srv1540371330=9, srv1553762281=11, srv590471835=22, srv1080081498=0, srv2003315676=14, srv2011181248=15, srv336359066=17, srv934295612=27, srv1663181123=13, srv691736341=24, srv1439296539=8, srv624508045=23, srv1213557031=1, srv1550101630=10, srv1336926690=6, srv986543697=29, srv1309778401=5, srv714730745=25, srv1248539515=2, srv530009806=20, srv2012765552=16} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-07 05:53:15,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-07 05:53:15,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-07 05:53:15,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-07 05:53:15,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-07 05:53:15,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-07 05:53:15,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-07 05:53:15,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-07 05:53:15,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-07 05:53:15,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-07 05:53:15,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-07 05:53:15,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-07 05:53:15,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-07 05:53:15,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-07 05:53:15,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-07 05:53:15,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-07 05:53:15,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-07 05:53:15,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-07 05:53:15,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-07 05:53:15,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-07 05:53:15,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-07 05:53:15,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-07 05:53:15,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-07 05:53:15,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-07 05:53:15,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-07 05:53:15,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-07 05:53:15,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-07 05:53:15,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-07 05:53:15,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-07 05:53:15,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-07 05:53:15,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-07 05:53:15,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-02-07 05:53:15,196 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0725393872567999, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-07 05:53:15,196 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912421612504898, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0725393872567999, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2024-02-07 05:55:15,170 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 7150205 different iterations.  Found a solution that moves 1841 regions; Going from a computed imbalance of 0.9912421612504898 to a new imbalance of 4.296985841319951E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6818518518518518, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-07 05:55:15,177 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1080081498:90 , srv1213557031:90 , srv1248539515:90 , srv128607315:90 , srv1296799755:90 , srv1309778401:90 , srv1336926690:90 , srv1404382841:90 , srv1439296539:90 , srv1540371330:90 , srv1550101630:90 , srv1553762281:90 , srv1572080445:90 , srv1663181123:90 , srv2003315676:90 , srv2011181248:90 , srv2012765552:90 , srv336359066:90 , srv410088987:90 , srv494976957:90 , srv530009806:90 , srv548930996:90 , srv590471835:90 , srv624508045:90 , srv691736341:90 , srv714730745:90 , srv757339386:90 , srv934295612:90 , srv958252600:90 , srv986543697:90 }
2024-02-07 05:55:15,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-07 05:55:15,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1404382841=7, srv757339386=26, srv410088987=18, srv958252600=28, srv128607315=3, srv494976957=19, srv1572080445=12, srv1296799755=4, srv548930996=21, srv1540371330=9, srv1553762281=11, srv590471835=22, srv1080081498=0, srv2003315676=14, srv2011181248=15, srv336359066=17, srv934295612=27, srv1663181123=13, srv691736341=24, srv1439296539=8, srv624508045=23, srv1213557031=1, srv1550101630=10, srv1336926690=6, srv986543697=29, srv1309778401=5, srv714730745=25, srv1248539515=2, srv530009806=20, srv2012765552=16} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-07 05:55:15,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-07 05:55:15,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-07 05:55:15,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-07 05:55:15,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-07 05:55:15,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-07 05:55:15,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-07 05:55:15,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-07 05:55:15,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-07 05:55:15,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-07 05:55:15,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-07 05:55:15,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-07 05:55:15,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-07 05:55:15,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-07 05:55:15,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-07 05:55:15,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-07 05:55:15,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-07 05:55:15,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-07 05:55:15,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-07 05:55:15,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-07 05:55:15,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-07 05:55:15,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-07 05:55:15,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-07 05:55:15,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-07 05:55:15,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-07 05:55:15,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-07 05:55:15,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-07 05:55:15,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-07 05:55:15,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-07 05:55:15,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-07 05:55:15,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-07 05:55:15,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-02-07 05:55:15,202 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-07 05:55:15,202 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1080081498:90 , srv1213557031:90 , srv1248539515:90 , srv128607315:90 , srv1296799755:90 , srv1309778401:90 , srv1336926690:90 , srv1404382841:90 , srv1439296539:90 , srv1540371330:90 , srv1550101630:90 , srv1553762281:90 , srv1572080445:90 , srv1663181123:90 , srv2003315676:90 , srv2011181248:90 , srv2012765552:90 , srv336359066:90 , srv410088987:90 , srv494976957:90 , srv530009806:90 , srv548930996:90 , srv590471835:90 , srv624508045:90 , srv691736341:90 , srv714730745:90 , srv757339386:90 , srv934295612:90 , srv958252600:90 , srv986543697:90 }
2024-02-07 05:55:15,223 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=196 (was 313), ProcessCount=169 (was 170), AvailableMemoryMB=3235 (was 2887) - AvailableMemoryMB LEAK? -