2023-02-19 17:53:57,633 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-02-19 17:53:58,004 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-02-19 17:53:58,165 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-02-19 17:53:58,250 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-19 17:53:58,258 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-02-19 17:53:58,273 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=309, ProcessCount=171, AvailableMemoryMB=10568
2023-02-19 17:53:58,276 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-19 17:53:58,276 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-02-19 17:53:58,419 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1033190489:84 , srv115862139:84 , srv1164318827:84 , srv1198166274:84 , srv1213238872:84 , srv1224627481:84 , srv1268570310:84 , srv13062740:84 , srv1309770330:84 , srv1320496016:84 , srv1367366184:84 , srv1389437344:84 , srv1394178865:84 , srv1425788764:84 , srv1438772554:84 , srv1475959686:84 , srv1483845996:84 , srv148868168:84 , srv1535992133:84 , srv1543968235:84 , srv1547084769:84 , srv1549398904:84 , srv1552699401:84 , srv157182947:84 , srv1602068834:84 , srv1613100266:84 , srv1626286820:84 , srv1626728047:84 , srv1656085877:84 , srv1686974875:84 , srv1760589705:84 , srv1771520109:84 , srv1779826214:84 , srv1792194671:84 , srv1809172277:84 , srv1813722663:84 , srv1821862860:84 , srv1840145104:84 , srv1870007020:84 , srv1922625661:84 , srv1928863454:84 , srv1943230340:84 , srv1946843711:84 , srv1960159060:84 , srv1989467187:84 , srv2000747083:84 , srv2006185435:84 , srv2020307534:84 , srv2062979116:84 , srv2081808366:84 , srv2089166135:84 , srv2093987329:84 , srv2124974100:84 , srv2139873781:84 , srv232200375:84 , srv235617841:84 , srv24559344:84 , srv251101675:84 , srv262914875:84 , srv27201505:84 , srv272151065:84 , srv273609535:84 , srv276811717:84 , srv292912282:84 , srv302233992:84 , srv349865992:84 , srv351044336:84 , srv374131184:84 , srv376861774:84 , srv389506877:84 , srv408282861:84 , srv413641473:84 , srv41437968:84 , srv425982092:84 , srv433902408:84 , srv456286701:84 , srv476678911:84 , srv4886255:84 , srv492250162:84 , srv502186692:84 , srv524448439:84 , srv542752235:84 , srv553027777:84 , srv563351702:84 , srv56717127:84 , srv621417726:84 , srv713040287:84 , srv724930494:84 , srv729815211:84 , srv737067747:84 , srv758977603:84 , srv850208880:84 , srv88615748:84 , srv923335117:84 , srv928607460:84 , srv950185859:84 , srv959136913:84 , srv962446907:84 , srv980824702:84 , srv69935110:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-02-19 17:53:58,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-19 17:53:58,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv724930494=88, srv235617841=55, srv1475959686=15, srv928607460=95, srv1367366184=10, srv553027777=82, srv232200375=54, srv1483845996=16, srv272151065=60, srv1656085877=28, srv959136913=97, srv1760589705=30, srv262914875=58, srv1960159060=43, srv1033190489=0, srv563351702=83, srv1771520109=31, srv713040287=87, srv1552699401=22, srv24559344=56, srv148868168=17, srv476678911=76, srv1928863454=40, srv923335117=94, srv980824702=99, srv1320496016=9, srv1686974875=29, srv1809172277=34, srv1602068834=24, srv1813722663=35, srv389506877=69, srv758977603=91, srv88615748=93, srv1626728047=27, srv349865992=65, srv1943230340=41, srv492250162=78, srv2124974100=52, srv737067747=90, srv351044336=66, srv1989467187=44, srv4886255=77, srv2089166135=50, srv56717127=84, srv502186692=79, srv1438772554=14, srv1626286820=26, srv962446907=98, srv376861774=68, srv2081808366=49, srv408282861=70, srv1213238872=4, srv2139873781=53, srv27201505=59, srv1164318827=2, srv157182947=23, srv1535992133=18, srv1543968235=19, srv1946843711=42, srv1394178865=12, srv1792194671=33, srv1309770330=8, srv2000747083=45, srv69935110=86, srv1268570310=6, srv542752235=81, srv1779826214=32, srv1549398904=21, srv2020307534=47, srv41437968=72, srv1425788764=13, srv621417726=85, srv374131184=67, srv2006185435=46, srv433902408=74, srv292912282=63, srv1389437344=11, srv1547084769=20, srv413641473=71, srv13062740=7, srv1198166274=3, srv1821862860=36, srv273609535=61, srv2093987329=51, srv456286701=75, srv302233992=64, srv1870007020=38, srv729815211=89, srv276811717=62, srv524448439=80, srv115862139=1, srv1224627481=5, srv850208880=92, srv2062979116=48, srv1922625661=39, srv425982092=73, srv1613100266=25, srv1840145104=37, srv950185859=96, srv251101675=57} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-19 17:53:58,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-19 17:53:58,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-19 17:53:58,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-19 17:53:58,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-19 17:53:58,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-19 17:53:58,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-19 17:53:58,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-19 17:53:58,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-19 17:53:58,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-19 17:53:58,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-19 17:53:58,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-19 17:53:58,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-19 17:53:58,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-19 17:53:58,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-19 17:53:58,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-19 17:53:58,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-19 17:53:58,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-19 17:53:58,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-19 17:53:58,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-19 17:53:58,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-19 17:53:58,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-19 17:53:58,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-19 17:53:58,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-19 17:53:58,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-19 17:53:58,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-19 17:53:58,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-19 17:53:58,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-19 17:53:58,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-19 17:53:58,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-19 17:53:58,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-19 17:53:58,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-19 17:53:58,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-19 17:53:58,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-19 17:53:58,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-19 17:53:58,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-19 17:53:58,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-19 17:53:58,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-19 17:53:58,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-19 17:53:58,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-19 17:53:58,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-19 17:53:58,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-19 17:53:58,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-19 17:53:58,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-19 17:53:58,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-19 17:53:58,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-19 17:53:58,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-19 17:53:58,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-19 17:53:58,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-19 17:53:58,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-19 17:53:58,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-19 17:53:58,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-19 17:53:58,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-19 17:53:58,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-19 17:53:58,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-19 17:53:58,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-19 17:53:58,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-19 17:53:58,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-19 17:53:58,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-19 17:53:58,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-19 17:53:58,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-19 17:53:58,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-19 17:53:58,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-19 17:53:58,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-19 17:53:58,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-19 17:53:58,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-19 17:53:58,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-19 17:53:58,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-19 17:53:58,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-19 17:53:58,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-19 17:53:58,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-19 17:53:58,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-19 17:53:58,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-19 17:53:58,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-19 17:53:58,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-19 17:53:58,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-19 17:53:58,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-19 17:53:58,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-19 17:53:58,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-19 17:53:58,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-19 17:53:58,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-19 17:53:58,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-19 17:53:58,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-19 17:53:58,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-19 17:53:58,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-19 17:53:58,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-19 17:53:58,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-19 17:53:58,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-19 17:53:58,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-19 17:53:58,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-19 17:53:58,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-19 17:53:58,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-19 17:53:58,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-19 17:53:58,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-19 17:53:58,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-19 17:53:58,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-19 17:53:58,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-19 17:53:58,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-19 17:53:58,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-19 17:53:58,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-19 17:53:58,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-19 17:53:58,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-19 17:53:59,033 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-19 17:53:59,034 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-19 17:53:59,034 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-19 17:53:59,369 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 769 ms to try 5000 different iterations.  Found a solution that moves 4655 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18150094933392077. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03754765853789705, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05707812591450825, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5172222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03754765853789705, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17666666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24416666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-19 17:53:59,394 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1771520109:83 , srv1779826214:83 , srv1821862860:83 , srv713040287:83 , srv980824702:83 , srv1626286820:84 , srv1813722663:84 , srv292912282:84 , srv389506877:84 , srv413641473:84 , srv88615748:84 , srv959136913:84 , srv1198166274:85 , srv1224627481:85 , srv1438772554:85 , srv1475959686:85 , srv1549398904:85 , srv1792194671:85 , srv276811717:85 , srv376861774:85 , srv408282861:85 , srv433902408:85 , srv4886255:85 , srv524448439:85 , srv923335117:85 , srv950185859:85 , srv1389437344:86 , srv1394178865:86 , srv1483845996:86 , srv1535992133:86 , srv1543968235:86 , srv1547084769:86 , srv1760589705:86 , srv1870007020:86 , srv1928863454:86 , srv1946843711:86 , srv2000747083:86 , srv2020307534:86 , srv2062979116:86 , srv2093987329:86 , srv235617841:86 , srv251101675:86 , srv273609535:86 , srv302233992:86 , srv349865992:86 , srv476678911:86 , srv502186692:86 , srv1033190489:87 , srv115862139:87 , srv1164318827:87 , srv1367366184:87 , srv148868168:87 , srv1602068834:87 , srv1626728047:87 , srv1656085877:87 , srv1922625661:87 , srv1989467187:87 , srv2089166135:87 , srv2139873781:87 , srv24559344:87 , srv262914875:87 , srv27201505:87 , srv272151065:87 , srv351044336:87 , srv374131184:87 , srv41437968:87 , srv553027777:87 , srv563351702:87 , srv56717127:87 , srv729815211:87 , srv758977603:87 , srv928607460:87 , srv962446907:87 , srv1309770330:88 , srv1320496016:88 , srv1425788764:88 , srv1613100266:88 , srv1686974875:88 , srv1943230340:88 , srv1960159060:88 , srv2124974100:88 , srv425982092:88 , srv492250162:88 , srv621417726:88 , srv737067747:88 , srv1213238872:89 , srv1268570310:89 , srv13062740:89 , srv157182947:89 , srv2006185435:89 , srv2081808366:89 , srv456286701:89 , srv1552699401:90 , srv1840145104:90 , srv542752235:90 , srv724930494:90 , srv850208880:90 , srv232200375:91 , srv1809172277:95 , srv69935110:424 }
2023-02-19 17:53:59,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-19 17:53:59,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv724930494=88, srv235617841=55, srv1475959686=15, srv928607460=95, srv1367366184=10, srv553027777=82, srv232200375=54, srv1483845996=16, srv272151065=60, srv1656085877=28, srv959136913=97, srv1760589705=30, srv262914875=58, srv1960159060=43, srv1033190489=0, srv563351702=83, srv1771520109=31, srv713040287=87, srv1552699401=22, srv24559344=56, srv148868168=17, srv476678911=76, srv1928863454=40, srv923335117=94, srv980824702=99, srv1320496016=9, srv1686974875=29, srv1809172277=34, srv1602068834=24, srv1813722663=35, srv389506877=69, srv758977603=91, srv88615748=93, srv1626728047=27, srv349865992=65, srv1943230340=41, srv492250162=78, srv2124974100=52, srv737067747=90, srv351044336=66, srv1989467187=44, srv4886255=77, srv2089166135=50, srv56717127=84, srv502186692=79, srv1438772554=14, srv1626286820=26, srv962446907=98, srv376861774=68, srv2081808366=49, srv408282861=70, srv1213238872=4, srv2139873781=53, srv27201505=59, srv1164318827=2, srv157182947=23, srv1535992133=18, srv1543968235=19, srv1946843711=42, srv1394178865=12, srv1792194671=33, srv1309770330=8, srv2000747083=45, srv69935110=86, srv1268570310=6, srv542752235=81, srv1779826214=32, srv1549398904=21, srv2020307534=47, srv41437968=72, srv1425788764=13, srv621417726=85, srv374131184=67, srv2006185435=46, srv433902408=74, srv292912282=63, srv1389437344=11, srv1547084769=20, srv413641473=71, srv13062740=7, srv1198166274=3, srv1821862860=36, srv273609535=61, srv2093987329=51, srv456286701=75, srv302233992=64, srv1870007020=38, srv729815211=89, srv276811717=62, srv524448439=80, srv115862139=1, srv1224627481=5, srv850208880=92, srv2062979116=48, srv1922625661=39, srv425982092=73, srv1613100266=25, srv1840145104=37, srv950185859=96, srv251101675=57} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-19 17:53:59,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-19 17:53:59,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-19 17:53:59,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-19 17:53:59,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-19 17:53:59,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-19 17:53:59,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-19 17:53:59,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-19 17:53:59,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-19 17:53:59,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-19 17:53:59,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-19 17:53:59,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-19 17:53:59,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-19 17:53:59,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-19 17:53:59,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-19 17:53:59,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-19 17:53:59,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-19 17:53:59,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-19 17:53:59,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-19 17:53:59,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-19 17:53:59,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-19 17:53:59,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-19 17:53:59,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-19 17:53:59,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-19 17:53:59,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-19 17:53:59,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-19 17:53:59,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-19 17:53:59,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-19 17:53:59,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-19 17:53:59,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-19 17:53:59,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-19 17:53:59,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-19 17:53:59,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-19 17:53:59,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-19 17:53:59,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-19 17:53:59,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-19 17:53:59,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-19 17:53:59,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-19 17:53:59,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-19 17:53:59,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-19 17:53:59,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-19 17:53:59,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-19 17:53:59,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-19 17:53:59,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-19 17:53:59,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-19 17:53:59,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-19 17:53:59,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-19 17:53:59,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-19 17:53:59,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-19 17:53:59,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-19 17:53:59,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-19 17:53:59,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-19 17:53:59,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-19 17:53:59,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-19 17:53:59,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-19 17:53:59,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-19 17:53:59,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-19 17:53:59,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-19 17:53:59,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-19 17:53:59,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-19 17:53:59,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-19 17:53:59,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-19 17:53:59,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-19 17:53:59,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-19 17:53:59,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-19 17:53:59,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-19 17:53:59,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-19 17:53:59,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-19 17:53:59,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-19 17:53:59,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-19 17:53:59,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-19 17:53:59,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-19 17:53:59,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-19 17:53:59,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-19 17:53:59,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-19 17:53:59,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-19 17:53:59,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-19 17:53:59,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-19 17:53:59,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-19 17:53:59,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-19 17:53:59,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-19 17:53:59,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-19 17:53:59,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-19 17:53:59,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-19 17:53:59,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-19 17:53:59,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-19 17:53:59,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-19 17:53:59,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-19 17:53:59,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-19 17:53:59,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-19 17:53:59,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-19 17:53:59,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-19 17:53:59,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-19 17:53:59,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-19 17:53:59,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-19 17:53:59,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-19 17:53:59,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-19 17:53:59,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-19 17:53:59,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-19 17:53:59,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-19 17:53:59,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-19 17:53:59,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-19 17:53:59,542 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03754765853789705, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05707812591450825, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03754765853789705, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17666666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24416666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-19 17:53:59,542 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-19 17:53:59,542 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18146835432725375, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03754765853789705, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05707812591450825, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03754765853789705, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17666666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24416666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-19 17:53:59,811 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 368 ms to try 5000 different iterations.  Found a solution that moves 2686 regions; Going from a computed imbalance of 0.18146835432725375 to a new imbalance of 0.01631752359657127. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.019719949759217138); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03139123527477564, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29844444444444446, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.019719949759217138); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04925, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-19 17:53:59,822 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1809172277:76 , srv1552699401:81 , srv1840145104:81 , srv232200375:81 , srv724930494:81 , srv850208880:81 , srv1268570310:82 , srv13062740:82 , srv157182947:82 , srv2006185435:82 , srv456286701:82 , srv621417726:82 , srv1213238872:83 , srv1320496016:83 , srv1870007020:83 , srv1922625661:83 , srv1943230340:83 , srv27201505:83 , srv542752235:83 , srv737067747:83 , srv1309770330:84 , srv1613100266:84 , srv1626728047:84 , srv1960159060:84 , srv2093987329:84 , srv2124974100:84 , srv235617841:84 , srv24559344:84 , srv349865992:84 , srv374131184:84 , srv492250162:84 , srv502186692:84 , srv56717127:84 , srv928607460:84 , srv115862139:85 , srv1425788764:85 , srv148868168:85 , srv1535992133:85 , srv1547084769:85 , srv2062979116:85 , srv273609535:85 , srv276811717:85 , srv351044336:85 , srv425982092:85 , srv553027777:85 , srv563351702:85 , srv1033190489:86 , srv1164318827:86 , srv1367366184:86 , srv1394178865:86 , srv1602068834:86 , srv1760589705:86 , srv1946843711:86 , srv2089166135:86 , srv272151065:86 , srv302233992:86 , srv408282861:86 , srv413641473:86 , srv41437968:86 , srv729815211:86 , srv1224627481:87 , srv1438772554:87 , srv1475959686:87 , srv1483845996:87 , srv1549398904:87 , srv1686974875:87 , srv1792194671:87 , srv1813722663:87 , srv1989467187:87 , srv2020307534:87 , srv2081808366:87 , srv262914875:87 , srv476678911:87 , srv524448439:87 , srv758977603:87 , srv923335117:87 , srv950185859:87 , srv980824702:87 , srv1198166274:88 , srv1389437344:88 , srv1543968235:88 , srv2000747083:88 , srv2139873781:88 , srv251101675:88 , srv433902408:88 , srv1656085877:89 , srv1771520109:89 , srv1821862860:89 , srv376861774:89 , srv389506877:89 , srv4886255:89 , srv88615748:89 , srv959136913:89 , srv1626286820:90 , srv292912282:90 , srv962446907:90 , srv1779826214:91 , srv713040287:91 , srv1928863454:92 , srv69935110:525 }
2023-02-19 17:53:59,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-19 17:53:59,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv724930494=88, srv235617841=55, srv1475959686=15, srv928607460=95, srv1367366184=10, srv553027777=82, srv232200375=54, srv1483845996=16, srv272151065=60, srv1656085877=28, srv959136913=97, srv1760589705=30, srv262914875=58, srv1960159060=43, srv1033190489=0, srv563351702=83, srv1771520109=31, srv713040287=87, srv1552699401=22, srv24559344=56, srv148868168=17, srv476678911=76, srv1928863454=40, srv923335117=94, srv980824702=99, srv1320496016=9, srv1686974875=29, srv1809172277=34, srv1602068834=24, srv1813722663=35, srv389506877=69, srv758977603=91, srv88615748=93, srv1626728047=27, srv349865992=65, srv1943230340=41, srv492250162=78, srv2124974100=52, srv737067747=90, srv351044336=66, srv1989467187=44, srv4886255=77, srv2089166135=50, srv56717127=84, srv502186692=79, srv1438772554=14, srv1626286820=26, srv962446907=98, srv376861774=68, srv2081808366=49, srv408282861=70, srv1213238872=4, srv2139873781=53, srv27201505=59, srv1164318827=2, srv157182947=23, srv1535992133=18, srv1543968235=19, srv1946843711=42, srv1394178865=12, srv1792194671=33, srv1309770330=8, srv2000747083=45, srv69935110=86, srv1268570310=6, srv542752235=81, srv1779826214=32, srv1549398904=21, srv2020307534=47, srv41437968=72, srv1425788764=13, srv621417726=85, srv374131184=67, srv2006185435=46, srv433902408=74, srv292912282=63, srv1389437344=11, srv1547084769=20, srv413641473=71, srv13062740=7, srv1198166274=3, srv1821862860=36, srv273609535=61, srv2093987329=51, srv456286701=75, srv302233992=64, srv1870007020=38, srv729815211=89, srv276811717=62, srv524448439=80, srv115862139=1, srv1224627481=5, srv850208880=92, srv2062979116=48, srv1922625661=39, srv425982092=73, srv1613100266=25, srv1840145104=37, srv950185859=96, srv251101675=57} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-19 17:53:59,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-19 17:53:59,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-19 17:53:59,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-19 17:53:59,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-19 17:53:59,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-19 17:53:59,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-19 17:53:59,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-19 17:53:59,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-19 17:53:59,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-19 17:53:59,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-19 17:53:59,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-19 17:53:59,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-19 17:53:59,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-19 17:53:59,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-19 17:53:59,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-19 17:53:59,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-19 17:53:59,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-19 17:53:59,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-19 17:53:59,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-19 17:53:59,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-19 17:53:59,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-19 17:53:59,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-19 17:53:59,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-19 17:53:59,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-19 17:53:59,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-19 17:53:59,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-19 17:53:59,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-19 17:53:59,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-19 17:53:59,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-19 17:53:59,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-19 17:53:59,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-19 17:53:59,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-19 17:53:59,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-19 17:53:59,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-19 17:53:59,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-19 17:53:59,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-19 17:53:59,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-19 17:53:59,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-19 17:53:59,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-19 17:53:59,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-19 17:53:59,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-19 17:53:59,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-19 17:53:59,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-19 17:53:59,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-19 17:53:59,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-19 17:53:59,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-19 17:53:59,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-19 17:53:59,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-19 17:53:59,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-19 17:53:59,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-19 17:53:59,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-19 17:53:59,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-19 17:53:59,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-19 17:53:59,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-19 17:53:59,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-19 17:53:59,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-19 17:53:59,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-19 17:53:59,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-19 17:53:59,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-19 17:53:59,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-19 17:53:59,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-19 17:53:59,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-19 17:53:59,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-19 17:53:59,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-19 17:53:59,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-19 17:53:59,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-19 17:53:59,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-19 17:53:59,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-19 17:53:59,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-19 17:53:59,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-19 17:53:59,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-19 17:53:59,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-19 17:53:59,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-19 17:53:59,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-19 17:53:59,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-19 17:53:59,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-19 17:53:59,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-19 17:53:59,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-19 17:53:59,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-19 17:53:59,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-19 17:53:59,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-19 17:53:59,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-19 17:53:59,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-19 17:53:59,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-19 17:53:59,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-19 17:53:59,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-19 17:53:59,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-19 17:53:59,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-19 17:53:59,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-19 17:53:59,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-19 17:53:59,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-19 17:53:59,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-19 17:53:59,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-19 17:53:59,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-19 17:53:59,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-19 17:53:59,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-19 17:53:59,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-19 17:53:59,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-19 17:53:59,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-19 17:53:59,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-19 17:53:59,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-19 17:53:59,964 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.019719949759217138); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03139123527477564, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.019719949759217138); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04925, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-19 17:53:59,965 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-19 17:53:59,965 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.016298715822584655, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.019719949759217138); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03139123527477564, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.019719949759217138); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04925, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-19 17:54:00,227 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 365 ms to try 5000 different iterations.  Found a solution that moves 1037 regions; Going from a computed imbalance of 0.016298715822584655 to a new imbalance of 5.477858752550546E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005868832712402046); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008800061218030102); MoveCostFunction : (multiplier=7.0, imbalance=0.11522222222222223, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005868832712402046); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0035833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-19 17:54:00,229 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1928863454:79 , srv1656085877:82 , srv2081808366:82 , srv962446907:82 , srv1389437344:83 , srv1543968235:83 , srv1626286820:83 , srv1686974875:83 , srv2139873781:83 , srv251101675:83 , srv262914875:83 , srv376861774:83 , srv713040287:83 , srv758977603:83 , srv1198166274:84 , srv1367366184:84 , srv1483845996:84 , srv1602068834:84 , srv1779826214:84 , srv2000747083:84 , srv2020307534:84 , srv2089166135:84 , srv292912282:84 , srv389506877:84 , srv433902408:84 , srv4886255:84 , srv542752235:84 , srv959136913:84 , srv1033190489:85 , srv115862139:85 , srv1164318827:85 , srv1213238872:85 , srv1425788764:85 , srv1438772554:85 , srv148868168:85 , srv1547084769:85 , srv1549398904:85 , srv1613100266:85 , srv1760589705:85 , srv1771520109:85 , srv1792194671:85 , srv1821862860:85 , srv1946843711:85 , srv1989467187:85 , srv232200375:85 , srv272151065:85 , srv302233992:85 , srv351044336:85 , srv41437968:85 , srv425982092:85 , srv476678911:85 , srv492250162:85 , srv729815211:85 , srv88615748:85 , srv1224627481:86 , srv1268570310:86 , srv1309770330:86 , srv1320496016:86 , srv1394178865:86 , srv1475959686:86 , srv1552699401:86 , srv1626728047:86 , srv1809172277:86 , srv1943230340:86 , srv1960159060:86 , srv2006185435:86 , srv2124974100:86 , srv24559344:86 , srv273609535:86 , srv374131184:86 , srv524448439:86 , srv553027777:86 , srv563351702:86 , srv56717127:86 , srv724930494:86 , srv850208880:86 , srv923335117:86 , srv950185859:86 , srv13062740:87 , srv1535992133:87 , srv157182947:87 , srv1813722663:87 , srv1840145104:87 , srv1922625661:87 , srv2062979116:87 , srv2093987329:87 , srv349865992:87 , srv408282861:87 , srv413641473:87 , srv456286701:87 , srv737067747:87 , srv928607460:87 , srv980824702:87 , srv235617841:88 , srv27201505:88 , srv276811717:88 , srv502186692:88 , srv621417726:88 , srv1870007020:89 , srv69935110:561 }
2023-02-19 17:54:00,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-19 17:54:00,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv724930494=88, srv235617841=55, srv1475959686=15, srv928607460=95, srv1367366184=10, srv553027777=82, srv232200375=54, srv1483845996=16, srv272151065=60, srv1656085877=28, srv959136913=97, srv1760589705=30, srv262914875=58, srv1960159060=43, srv1033190489=0, srv563351702=83, srv1771520109=31, srv713040287=87, srv1552699401=22, srv24559344=56, srv148868168=17, srv476678911=76, srv1928863454=40, srv923335117=94, srv980824702=99, srv1320496016=9, srv1686974875=29, srv1809172277=34, srv1602068834=24, srv1813722663=35, srv389506877=69, srv758977603=91, srv88615748=93, srv1626728047=27, srv349865992=65, srv1943230340=41, srv492250162=78, srv2124974100=52, srv737067747=90, srv351044336=66, srv1989467187=44, srv4886255=77, srv2089166135=50, srv56717127=84, srv502186692=79, srv1438772554=14, srv1626286820=26, srv962446907=98, srv376861774=68, srv2081808366=49, srv408282861=70, srv1213238872=4, srv2139873781=53, srv27201505=59, srv1164318827=2, srv157182947=23, srv1535992133=18, srv1543968235=19, srv1946843711=42, srv1394178865=12, srv1792194671=33, srv1309770330=8, srv2000747083=45, srv69935110=86, srv1268570310=6, srv542752235=81, srv1779826214=32, srv1549398904=21, srv2020307534=47, srv41437968=72, srv1425788764=13, srv621417726=85, srv374131184=67, srv2006185435=46, srv433902408=74, srv292912282=63, srv1389437344=11, srv1547084769=20, srv413641473=71, srv13062740=7, srv1198166274=3, srv1821862860=36, srv273609535=61, srv2093987329=51, srv456286701=75, srv302233992=64, srv1870007020=38, srv729815211=89, srv276811717=62, srv524448439=80, srv115862139=1, srv1224627481=5, srv850208880=92, srv2062979116=48, srv1922625661=39, srv425982092=73, srv1613100266=25, srv1840145104=37, srv950185859=96, srv251101675=57} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-19 17:54:00,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-19 17:54:00,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-19 17:54:00,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-19 17:54:00,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-19 17:54:00,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-19 17:54:00,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-19 17:54:00,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-19 17:54:00,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-19 17:54:00,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-19 17:54:00,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-19 17:54:00,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-19 17:54:00,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-19 17:54:00,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-19 17:54:00,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-19 17:54:00,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-19 17:54:00,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-19 17:54:00,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-19 17:54:00,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-19 17:54:00,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-19 17:54:00,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-19 17:54:00,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-19 17:54:00,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-19 17:54:00,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-19 17:54:00,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-19 17:54:00,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-19 17:54:00,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-19 17:54:00,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-19 17:54:00,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-19 17:54:00,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-19 17:54:00,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-19 17:54:00,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-19 17:54:00,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-19 17:54:00,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-19 17:54:00,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-19 17:54:00,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-19 17:54:00,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-19 17:54:00,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-19 17:54:00,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-19 17:54:00,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-19 17:54:00,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-19 17:54:00,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-19 17:54:00,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-19 17:54:00,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-19 17:54:00,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-19 17:54:00,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-19 17:54:00,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-19 17:54:00,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-19 17:54:00,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-19 17:54:00,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-19 17:54:00,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-19 17:54:00,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-19 17:54:00,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-19 17:54:00,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-19 17:54:00,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-19 17:54:00,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-19 17:54:00,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-19 17:54:00,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-19 17:54:00,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-19 17:54:00,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-19 17:54:00,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-19 17:54:00,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-19 17:54:00,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-19 17:54:00,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-19 17:54:00,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-19 17:54:00,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-19 17:54:00,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-19 17:54:00,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-19 17:54:00,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-19 17:54:00,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-19 17:54:00,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-19 17:54:00,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-19 17:54:00,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-19 17:54:00,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-19 17:54:00,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-19 17:54:00,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-19 17:54:00,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-19 17:54:00,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-19 17:54:00,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-19 17:54:00,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-19 17:54:00,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-19 17:54:00,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-19 17:54:00,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-19 17:54:00,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-19 17:54:00,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-19 17:54:00,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-19 17:54:00,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-19 17:54:00,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-19 17:54:00,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-19 17:54:00,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-19 17:54:00,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-19 17:54:00,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-19 17:54:00,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-19 17:54:00,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-19 17:54:00,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-19 17:54:00,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-19 17:54:00,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-19 17:54:00,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-19 17:54:00,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-19 17:54:00,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-19 17:54:00,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-19 17:54:00,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-19 17:54:00,378 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005868832712402046); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008800061218030102); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005868832712402046); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0035833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-19 17:54:00,379 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-19 17:54:00,379 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=5.405246460576911E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005868832712402046); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008800061218030102); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005868832712402046); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0035833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-19 17:54:00,732 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 453 ms to try 5000 different iterations.  Found a solution that moves 249 regions; Going from a computed imbalance of 5.405246460576911E-4 to a new imbalance of 1.9630913218651676E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017727200782610348); MoveCostFunction : (multiplier=7.0, imbalance=0.027666666666666666, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-19 17:54:00,733 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1164318827:83 , srv1224627481:83 , srv1870007020:83 , srv923335117:83 , srv962446907:83 , srv1033190489:84 , srv13062740:84 , srv1309770330:84 , srv1394178865:84 , srv1438772554:84 , srv1475959686:84 , srv1535992133:84 , srv157182947:84 , srv1613100266:84 , srv1656085877:84 , srv1686974875:84 , srv1779826214:84 , srv1813722663:84 , srv1840145104:84 , srv1960159060:84 , srv1989467187:84 , srv2000747083:84 , srv2062979116:84 , srv2081808366:84 , srv2124974100:84 , srv2139873781:84 , srv262914875:84 , srv27201505:84 , srv272151065:84 , srv273609535:84 , srv276811717:84 , srv292912282:84 , srv349865992:84 , srv408282861:84 , srv41437968:84 , srv456286701:84 , srv476678911:84 , srv4886255:84 , srv492250162:84 , srv502186692:84 , srv524448439:84 , srv553027777:84 , srv563351702:84 , srv621417726:84 , srv737067747:84 , srv88615748:84 , srv950185859:84 , srv1198166274:85 , srv1213238872:85 , srv1268570310:85 , srv1320496016:85 , srv1367366184:85 , srv1389437344:85 , srv1425788764:85 , srv1483845996:85 , srv148868168:85 , srv1543968235:85 , srv1549398904:85 , srv1552699401:85 , srv1602068834:85 , srv1626286820:85 , srv1626728047:85 , srv1760589705:85 , srv1771520109:85 , srv1792194671:85 , srv1809172277:85 , srv1821862860:85 , srv1922625661:85 , srv1928863454:85 , srv1943230340:85 , srv1946843711:85 , srv2006185435:85 , srv2020307534:85 , srv2089166135:85 , srv2093987329:85 , srv232200375:85 , srv235617841:85 , srv24559344:85 , srv251101675:85 , srv302233992:85 , srv351044336:85 , srv374131184:85 , srv376861774:85 , srv389506877:85 , srv413641473:85 , srv425982092:85 , srv433902408:85 , srv724930494:85 , srv729815211:85 , srv758977603:85 , srv850208880:85 , srv928607460:85 , srv959136913:85 , srv980824702:85 , srv115862139:86 , srv1547084769:86 , srv542752235:86 , srv56717127:86 , srv713040287:86 , srv69935110:632 }
2023-02-19 17:54:00,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-19 17:54:00,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv724930494=88, srv235617841=55, srv1475959686=15, srv928607460=95, srv1367366184=10, srv553027777=82, srv232200375=54, srv1483845996=16, srv272151065=60, srv1656085877=28, srv959136913=97, srv1760589705=30, srv262914875=58, srv1960159060=43, srv1033190489=0, srv563351702=83, srv1771520109=31, srv713040287=87, srv1552699401=22, srv24559344=56, srv148868168=17, srv476678911=76, srv1928863454=40, srv923335117=94, srv980824702=99, srv1320496016=9, srv1686974875=29, srv1809172277=34, srv1602068834=24, srv1813722663=35, srv389506877=69, srv758977603=91, srv88615748=93, srv1626728047=27, srv349865992=65, srv1943230340=41, srv492250162=78, srv2124974100=52, srv737067747=90, srv351044336=66, srv1989467187=44, srv4886255=77, srv2089166135=50, srv56717127=84, srv502186692=79, srv1438772554=14, srv1626286820=26, srv962446907=98, srv376861774=68, srv2081808366=49, srv408282861=70, srv1213238872=4, srv2139873781=53, srv27201505=59, srv1164318827=2, srv157182947=23, srv1535992133=18, srv1543968235=19, srv1946843711=42, srv1394178865=12, srv1792194671=33, srv1309770330=8, srv2000747083=45, srv69935110=86, srv1268570310=6, srv542752235=81, srv1779826214=32, srv1549398904=21, srv2020307534=47, srv41437968=72, srv1425788764=13, srv621417726=85, srv374131184=67, srv2006185435=46, srv433902408=74, srv292912282=63, srv1389437344=11, srv1547084769=20, srv413641473=71, srv13062740=7, srv1198166274=3, srv1821862860=36, srv273609535=61, srv2093987329=51, srv456286701=75, srv302233992=64, srv1870007020=38, srv729815211=89, srv276811717=62, srv524448439=80, srv115862139=1, srv1224627481=5, srv850208880=92, srv2062979116=48, srv1922625661=39, srv425982092=73, srv1613100266=25, srv1840145104=37, srv950185859=96, srv251101675=57} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-19 17:54:00,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-19 17:54:00,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-19 17:54:00,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-19 17:54:00,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-19 17:54:00,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-19 17:54:00,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-19 17:54:00,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-19 17:54:00,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-19 17:54:00,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-19 17:54:00,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-19 17:54:00,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-19 17:54:00,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-19 17:54:00,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-19 17:54:00,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-19 17:54:00,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-19 17:54:00,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-19 17:54:00,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-19 17:54:00,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-19 17:54:00,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-19 17:54:00,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-19 17:54:00,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-19 17:54:00,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-19 17:54:00,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-19 17:54:00,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-19 17:54:00,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-19 17:54:00,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-19 17:54:00,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-19 17:54:00,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-19 17:54:00,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-19 17:54:00,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-19 17:54:00,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-19 17:54:00,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-19 17:54:00,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-19 17:54:00,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-19 17:54:00,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-19 17:54:00,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-19 17:54:00,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-19 17:54:00,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-19 17:54:00,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-19 17:54:00,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-19 17:54:00,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-19 17:54:00,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-19 17:54:00,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-19 17:54:00,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-19 17:54:00,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-19 17:54:00,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-19 17:54:00,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-19 17:54:00,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-19 17:54:00,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-19 17:54:00,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-19 17:54:00,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-19 17:54:00,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-19 17:54:00,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-19 17:54:00,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-19 17:54:00,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-19 17:54:00,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-19 17:54:00,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-19 17:54:00,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-19 17:54:00,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-19 17:54:00,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-19 17:54:00,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-19 17:54:00,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-19 17:54:00,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-19 17:54:00,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-19 17:54:00,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-19 17:54:00,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-19 17:54:00,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-19 17:54:00,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-19 17:54:00,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-19 17:54:00,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-19 17:54:00,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-19 17:54:00,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-19 17:54:00,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-19 17:54:00,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-19 17:54:00,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-19 17:54:00,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-19 17:54:00,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-19 17:54:00,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-19 17:54:00,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-19 17:54:00,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-19 17:54:00,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-19 17:54:00,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-19 17:54:00,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-19 17:54:00,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-19 17:54:00,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-19 17:54:00,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-19 17:54:00,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-19 17:54:00,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-19 17:54:00,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-19 17:54:00,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-19 17:54:00,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-19 17:54:00,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-19 17:54:00,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-19 17:54:00,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-19 17:54:00,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-19 17:54:00,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-19 17:54:00,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-19 17:54:00,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-19 17:54:00,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-19 17:54:00,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-19 17:54:00,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-19 17:54:00,885 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.7887377953325223E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017727200782610348); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-19 17:54:00,885 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1164318827:83 , srv1224627481:83 , srv1870007020:83 , srv923335117:83 , srv962446907:83 , srv1033190489:84 , srv13062740:84 , srv1309770330:84 , srv1394178865:84 , srv1438772554:84 , srv1475959686:84 , srv1535992133:84 , srv157182947:84 , srv1613100266:84 , srv1656085877:84 , srv1686974875:84 , srv1779826214:84 , srv1813722663:84 , srv1840145104:84 , srv1960159060:84 , srv1989467187:84 , srv2000747083:84 , srv2062979116:84 , srv2081808366:84 , srv2124974100:84 , srv2139873781:84 , srv262914875:84 , srv27201505:84 , srv272151065:84 , srv273609535:84 , srv276811717:84 , srv292912282:84 , srv349865992:84 , srv408282861:84 , srv41437968:84 , srv456286701:84 , srv476678911:84 , srv4886255:84 , srv492250162:84 , srv502186692:84 , srv524448439:84 , srv553027777:84 , srv563351702:84 , srv621417726:84 , srv737067747:84 , srv88615748:84 , srv950185859:84 , srv1198166274:85 , srv1213238872:85 , srv1268570310:85 , srv1320496016:85 , srv1367366184:85 , srv1389437344:85 , srv1425788764:85 , srv1483845996:85 , srv148868168:85 , srv1543968235:85 , srv1549398904:85 , srv1552699401:85 , srv1602068834:85 , srv1626286820:85 , srv1626728047:85 , srv1760589705:85 , srv1771520109:85 , srv1792194671:85 , srv1809172277:85 , srv1821862860:85 , srv1922625661:85 , srv1928863454:85 , srv1943230340:85 , srv1946843711:85 , srv2006185435:85 , srv2020307534:85 , srv2089166135:85 , srv2093987329:85 , srv232200375:85 , srv235617841:85 , srv24559344:85 , srv251101675:85 , srv302233992:85 , srv351044336:85 , srv374131184:85 , srv376861774:85 , srv389506877:85 , srv413641473:85 , srv425982092:85 , srv433902408:85 , srv724930494:85 , srv729815211:85 , srv758977603:85 , srv850208880:85 , srv928607460:85 , srv959136913:85 , srv980824702:85 , srv115862139:86 , srv1547084769:86 , srv542752235:86 , srv56717127:86 , srv713040287:86 , srv69935110:632 }
2023-02-19 17:54:00,922 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=356 (was 309) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 171), AvailableMemoryMB=10106 (was 10568)
2023-02-19 17:54:00,929 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=356, ProcessCount=170, AvailableMemoryMB=10106
2023-02-19 17:54:00,930 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-19 17:54:00,931 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-02-19 17:54:00,948 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1006533035:84 , srv1149239836:84 , srv1178384214:84 , srv1239482687:84 , srv1310227884:84 , srv1338996714:84 , srv1410494893:84 , srv1413408615:84 , srv1416861254:84 , srv1456551770:84 , srv1475674408:84 , srv1626908034:84 , srv1639272970:84 , srv1737833031:84 , srv1737926029:84 , srv1803868621:84 , srv1944012769:84 , srv2064875623:84 , srv272099014:84 , srv339393223:84 , srv415049562:84 , srv691514908:84 , srv747878251:84 , srv78822299:84 , srv802099078:84 , srv813026319:84 , srv944946252:84 , srv973822392:84 , srv992932931:84 , srv2131626866:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-02-19 17:54:00,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-19 17:54:00,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1178384214=2, srv1944012769=16, srv339393223=20, srv1338996714=5, srv1737926029=14, srv1410494893=6, srv1310227884=4, srv2131626866=18, srv78822299=24, srv1239482687=3, srv1413408615=7, srv415049562=21, srv2064875623=17, srv1639272970=12, srv1149239836=1, srv1737833031=13, srv813026319=26, srv1006533035=0, srv1416861254=8, srv944946252=27, srv1456551770=9, srv272099014=19, srv1475674408=10, srv802099078=25, srv747878251=23, srv973822392=28, srv992932931=29, srv1803868621=15, srv1626908034=11, srv691514908=22} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-19 17:54:00,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-19 17:54:00,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-19 17:54:00,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-19 17:54:00,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-19 17:54:00,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-19 17:54:00,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-19 17:54:00,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-19 17:54:00,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-19 17:54:00,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-19 17:54:00,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-19 17:54:00,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-19 17:54:00,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-19 17:54:00,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-19 17:54:00,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-19 17:54:00,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-19 17:54:00,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-19 17:54:00,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-19 17:54:00,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-19 17:54:00,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-19 17:54:00,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-19 17:54:00,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-19 17:54:00,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-19 17:54:00,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-19 17:54:00,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-19 17:54:00,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-19 17:54:00,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-19 17:54:00,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-19 17:54:00,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-19 17:54:00,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-19 17:54:00,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-19 17:54:00,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-02-19 17:54:00,994 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0567420182880445, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-19 17:54:00,994 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912371835511109, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0567420182880445, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-02-19 17:56:00,975 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5827117 different iterations.  Found a solution that moves 1856 regions; Going from a computed imbalance of 0.9912371835511109 to a new imbalance of 4.3319965896196796E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6874074074074074, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-19 17:56:00,983 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1006533035:90 , srv1149239836:90 , srv1178384214:90 , srv1239482687:90 , srv1310227884:90 , srv1338996714:90 , srv1410494893:90 , srv1413408615:90 , srv1416861254:90 , srv1456551770:90 , srv1475674408:90 , srv1626908034:90 , srv1639272970:90 , srv1737833031:90 , srv1737926029:90 , srv1803868621:90 , srv1944012769:90 , srv2064875623:90 , srv2131626866:90 , srv272099014:90 , srv339393223:90 , srv415049562:90 , srv691514908:90 , srv747878251:90 , srv78822299:90 , srv802099078:90 , srv813026319:90 , srv944946252:90 , srv973822392:90 , srv992932931:90 }
2023-02-19 17:56:00,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-19 17:56:00,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1178384214=2, srv1944012769=16, srv339393223=20, srv1338996714=5, srv1737926029=14, srv1410494893=6, srv1310227884=4, srv2131626866=18, srv78822299=24, srv1239482687=3, srv1413408615=7, srv415049562=21, srv2064875623=17, srv1639272970=12, srv1149239836=1, srv1737833031=13, srv813026319=26, srv1006533035=0, srv1416861254=8, srv944946252=27, srv1456551770=9, srv272099014=19, srv1475674408=10, srv802099078=25, srv747878251=23, srv973822392=28, srv992932931=29, srv1803868621=15, srv1626908034=11, srv691514908=22} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-19 17:56:00,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-19 17:56:00,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-19 17:56:00,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-19 17:56:00,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-19 17:56:00,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-19 17:56:00,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-19 17:56:00,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-19 17:56:00,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-19 17:56:00,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-19 17:56:00,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-19 17:56:00,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-19 17:56:00,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-19 17:56:00,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-19 17:56:00,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-19 17:56:00,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-19 17:56:00,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-19 17:56:00,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-19 17:56:00,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-19 17:56:00,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-19 17:56:00,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-19 17:56:00,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-19 17:56:00,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-19 17:56:00,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-19 17:56:00,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-19 17:56:00,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-19 17:56:00,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-19 17:56:00,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-19 17:56:00,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-19 17:56:00,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-19 17:56:00,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-19 17:56:00,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-02-19 17:56:01,008 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-19 17:56:01,008 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1006533035:90 , srv1149239836:90 , srv1178384214:90 , srv1239482687:90 , srv1310227884:90 , srv1338996714:90 , srv1410494893:90 , srv1413408615:90 , srv1416861254:90 , srv1456551770:90 , srv1475674408:90 , srv1626908034:90 , srv1639272970:90 , srv1737833031:90 , srv1737926029:90 , srv1803868621:90 , srv1944012769:90 , srv2064875623:90 , srv2131626866:90 , srv272099014:90 , srv339393223:90 , srv415049562:90 , srv691514908:90 , srv747878251:90 , srv78822299:90 , srv802099078:90 , srv813026319:90 , srv944946252:90 , srv973822392:90 , srv992932931:90 }
2023-02-19 17:56:01,022 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=248 (was 356), ProcessCount=170 (was 170), AvailableMemoryMB=9883 (was 10106)