2023-02-23 12:53:59,412 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-02-23 12:53:59,726 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-02-23 12:53:59,860 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-02-23 12:53:59,923 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-23 12:53:59,930 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-02-23 12:53:59,946 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=286, ProcessCount=170, AvailableMemoryMB=6589
2023-02-23 12:53:59,949 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-23 12:53:59,949 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-02-23 12:54:00,087 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1044595535:84 , srv1068815799:84 , srv1093633209:84 , srv1098629370:84 , srv1106956988:84 , srv1116767657:84 , srv112334816:84 , srv1129778566:84 , srv1131885373:84 , srv1172900549:84 , srv1201555398:84 , srv1249829012:84 , srv1254958702:84 , srv1260379304:84 , srv127521909:84 , srv1282086934:84 , srv1296874411:84 , srv1298044022:84 , srv1320548417:84 , srv1363114969:84 , srv1385508202:84 , srv1410262742:84 , srv1412033070:84 , srv1413638516:84 , srv1424056366:84 , srv1448136735:84 , srv1484777926:84 , srv1554647824:84 , srv1554855276:84 , srv1559962570:84 , srv15637876:84 , srv159746081:84 , srv1605857888:84 , srv1646945984:84 , srv1681369271:84 , srv1694361171:84 , srv1706430659:84 , srv1708436565:84 , srv1714594973:84 , srv1717740466:84 , srv1730051979:84 , srv1749357228:84 , srv1768383670:84 , srv1780152375:84 , srv1789907047:84 , srv1807611876:84 , srv1929506318:84 , srv1929813241:84 , srv1938400004:84 , srv1938867924:84 , srv1940149584:84 , srv1941901387:84 , srv1951466482:84 , srv1981082403:84 , srv2047904159:84 , srv2060117409:84 , srv2090829640:84 , srv2105747058:84 , srv2115776532:84 , srv247801399:84 , srv250121446:84 , srv263529034:84 , srv290907532:84 , srv354264732:84 , srv355414864:84 , srv397517396:84 , srv41972961:84 , srv434402727:84 , srv480794400:84 , srv481335518:84 , srv503015366:84 , srv516938821:84 , srv524358192:84 , srv52492372:84 , srv53736856:84 , srv542644028:84 , srv571270375:84 , srv622795080:84 , srv635654725:84 , srv66142673:84 , srv663549327:84 , srv691043731:84 , srv697554800:84 , srv698033253:84 , srv703713637:84 , srv743336876:84 , srv770282871:84 , srv784409722:84 , srv813260134:84 , srv81633427:84 , srv819325586:84 , srv823031647:84 , srv826455605:84 , srv835665298:84 , srv902934368:84 , srv923086408:84 , srv941739914:84 , srv950071303:84 , srv971518830:84 , srv1825796688:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-02-23 12:54:00,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-23 12:54:00,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1385508202=20, srv1412033070=22, srv159746081=31, srv354264732=64, srv835665298=94, srv355414864=65, srv826455605=93, srv1044595535=0, srv112334816=6, srv1448136735=25, srv15637876=30, srv1131885373=8, srv697554800=83, srv971518830=99, srv1282086934=15, srv770282871=87, srv950071303=98, srv1825796688=46, srv902934368=95, srv1559962570=29, srv703713637=85, srv1938867924=50, srv1068815799=1, srv1941901387=52, srv1172900549=9, srv503015366=71, srv1260379304=13, srv1768383670=42, srv41972961=67, srv1929506318=47, srv52492372=74, srv1554647824=27, srv247801399=60, srv1413638516=23, srv2115776532=59, srv923086408=96, srv516938821=72, srv743336876=86, srv524358192=73, srv1681369271=34, srv1106956988=4, srv1780152375=43, srv263529034=62, srv290907532=63, srv1254958702=12, srv1981082403=54, srv819325586=91, srv1129778566=7, srv1708436565=37, srv1706430659=36, srv698033253=84, srv1484777926=26, srv1093633209=2, srv81633427=90, srv823031647=92, srv1951466482=53, srv1730051979=40, srv480794400=69, srv1807611876=45, srv1098629370=3, srv1363114969=19, srv1717740466=39, srv1714594973=38, srv663549327=81, srv941739914=97, srv1249829012=11, srv397517396=66, srv1940149584=51, srv1929813241=48, srv127521909=14, srv784409722=88, srv1694361171=35, srv2105747058=58, srv1749357228=41, srv571270375=77, srv2060117409=56, srv2090829640=57, srv1938400004=49, srv481335518=70, srv635654725=79, srv1298044022=17, srv1424056366=24, srv1554855276=28, srv622795080=78, srv1296874411=16, srv542644028=76, srv250121446=61, srv66142673=80, srv1605857888=32, srv53736856=75, srv2047904159=55, srv1320548417=18, srv434402727=68, srv813260134=89, srv1201555398=10, srv1789907047=44, srv1116767657=5, srv691043731=82, srv1410262742=21, srv1646945984=33} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-23 12:54:00,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-23 12:54:00,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-23 12:54:00,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-23 12:54:00,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-23 12:54:00,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-23 12:54:00,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-23 12:54:00,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-23 12:54:00,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-23 12:54:00,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-23 12:54:00,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-23 12:54:00,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-23 12:54:00,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-23 12:54:00,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-23 12:54:00,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-23 12:54:00,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-23 12:54:00,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-23 12:54:00,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-23 12:54:00,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-23 12:54:00,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-23 12:54:00,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-23 12:54:00,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-23 12:54:00,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-23 12:54:00,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-23 12:54:00,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-23 12:54:00,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-23 12:54:00,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-23 12:54:00,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-23 12:54:00,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-23 12:54:00,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-23 12:54:00,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-23 12:54:00,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-23 12:54:00,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-23 12:54:00,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-23 12:54:00,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-23 12:54:00,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-23 12:54:00,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-23 12:54:00,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-23 12:54:00,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-23 12:54:00,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-23 12:54:00,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-23 12:54:00,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-23 12:54:00,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-23 12:54:00,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-23 12:54:00,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-23 12:54:00,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-23 12:54:00,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-23 12:54:00,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-23 12:54:00,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-23 12:54:00,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-23 12:54:00,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-23 12:54:00,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-23 12:54:00,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-23 12:54:00,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-23 12:54:00,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-23 12:54:00,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-23 12:54:00,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-23 12:54:00,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-23 12:54:00,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-23 12:54:00,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-23 12:54:00,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-23 12:54:00,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-23 12:54:00,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-23 12:54:00,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-23 12:54:00,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-23 12:54:00,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-23 12:54:00,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-23 12:54:00,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-23 12:54:00,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-23 12:54:00,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-23 12:54:00,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-23 12:54:00,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-23 12:54:00,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-23 12:54:00,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-23 12:54:00,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-23 12:54:00,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-23 12:54:00,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-23 12:54:00,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-23 12:54:00,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-23 12:54:00,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-23 12:54:00,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-23 12:54:00,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-23 12:54:00,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-23 12:54:00,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-23 12:54:00,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-23 12:54:00,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-23 12:54:00,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-23 12:54:00,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-23 12:54:00,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-23 12:54:00,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-23 12:54:00,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-23 12:54:00,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-23 12:54:00,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-23 12:54:00,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-23 12:54:00,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-23 12:54:00,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-23 12:54:00,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-23 12:54:00,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-23 12:54:00,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-23 12:54:00,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-23 12:54:00,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-23 12:54:00,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-23 12:54:00,401 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-23 12:54:00,401 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-23 12:54:00,402 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-23 12:54:00,696 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 438 ms to try 5000 different iterations.  Found a solution that moves 4709 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1790777758144065. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03951255560111164, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05490732933602594, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5232222222222223, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03951255560111164, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17416666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24225, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-23 12:54:00,706 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1938400004:80 , srv2090829640:80 , srv826455605:81 , srv1098629370:82 , srv1448136735:82 , srv571270375:82 , srv1116767657:83 , srv1554855276:83 , srv1708436565:83 , srv112334816:84 , srv1282086934:84 , srv1554647824:84 , srv159746081:84 , srv1730051979:84 , srv1789907047:84 , srv1929506318:84 , srv1929813241:84 , srv2047904159:84 , srv2060117409:84 , srv263529034:84 , srv941739914:84 , srv1068815799:85 , srv1093633209:85 , srv1260379304:85 , srv1296874411:85 , srv1410262742:85 , srv1559962570:85 , srv1605857888:85 , srv1780152375:85 , srv1807611876:85 , srv250121446:85 , srv481335518:85 , srv52492372:85 , srv698033253:85 , srv902934368:85 , srv1106956988:86 , srv1172900549:86 , srv1254958702:86 , srv1646945984:86 , srv1714594973:86 , srv1941901387:86 , srv1951466482:86 , srv355414864:86 , srv434402727:86 , srv480794400:86 , srv53736856:86 , srv66142673:86 , srv663549327:86 , srv784409722:86 , srv823031647:86 , srv835665298:86 , srv950071303:86 , srv1129778566:87 , srv1131885373:87 , srv1201555398:87 , srv1249829012:87 , srv1424056366:87 , srv1717740466:87 , srv1768383670:87 , srv1940149584:87 , srv1981082403:87 , srv2115776532:87 , srv290907532:87 , srv354264732:87 , srv503015366:87 , srv622795080:87 , srv635654725:87 , srv703713637:87 , srv923086408:87 , srv971518830:87 , srv1298044022:88 , srv1320548417:88 , srv1413638516:88 , srv1484777926:88 , srv1694361171:88 , srv1749357228:88 , srv2105747058:88 , srv247801399:88 , srv41972961:88 , srv516938821:88 , srv691043731:88 , srv1363114969:89 , srv1385508202:89 , srv15637876:89 , srv1706430659:89 , srv524358192:89 , srv743336876:89 , srv813260134:89 , srv81633427:89 , srv819325586:89 , srv1412033070:90 , srv397517396:90 , srv542644028:90 , srv770282871:90 , srv697554800:91 , srv1044595535:92 , srv1681369271:92 , srv127521909:94 , srv1938867924:99 , srv1825796688:441 }
2023-02-23 12:54:00,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-23 12:54:00,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1385508202=20, srv1412033070=22, srv159746081=31, srv354264732=64, srv835665298=94, srv355414864=65, srv826455605=93, srv1044595535=0, srv112334816=6, srv1448136735=25, srv15637876=30, srv1131885373=8, srv697554800=83, srv971518830=99, srv1282086934=15, srv770282871=87, srv950071303=98, srv1825796688=46, srv902934368=95, srv1559962570=29, srv703713637=85, srv1938867924=50, srv1068815799=1, srv1941901387=52, srv1172900549=9, srv503015366=71, srv1260379304=13, srv1768383670=42, srv41972961=67, srv1929506318=47, srv52492372=74, srv1554647824=27, srv247801399=60, srv1413638516=23, srv2115776532=59, srv923086408=96, srv516938821=72, srv743336876=86, srv524358192=73, srv1681369271=34, srv1106956988=4, srv1780152375=43, srv263529034=62, srv290907532=63, srv1254958702=12, srv1981082403=54, srv819325586=91, srv1129778566=7, srv1708436565=37, srv1706430659=36, srv698033253=84, srv1484777926=26, srv1093633209=2, srv81633427=90, srv823031647=92, srv1951466482=53, srv1730051979=40, srv480794400=69, srv1807611876=45, srv1098629370=3, srv1363114969=19, srv1717740466=39, srv1714594973=38, srv663549327=81, srv941739914=97, srv1249829012=11, srv397517396=66, srv1940149584=51, srv1929813241=48, srv127521909=14, srv784409722=88, srv1694361171=35, srv2105747058=58, srv1749357228=41, srv571270375=77, srv2060117409=56, srv2090829640=57, srv1938400004=49, srv481335518=70, srv635654725=79, srv1298044022=17, srv1424056366=24, srv1554855276=28, srv622795080=78, srv1296874411=16, srv542644028=76, srv250121446=61, srv66142673=80, srv1605857888=32, srv53736856=75, srv2047904159=55, srv1320548417=18, srv434402727=68, srv813260134=89, srv1201555398=10, srv1789907047=44, srv1116767657=5, srv691043731=82, srv1410262742=21, srv1646945984=33} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-23 12:54:00,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-23 12:54:00,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-23 12:54:00,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-23 12:54:00,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-23 12:54:00,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-23 12:54:00,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-23 12:54:00,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-23 12:54:00,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-23 12:54:00,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-23 12:54:00,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-23 12:54:00,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-23 12:54:00,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-23 12:54:00,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-23 12:54:00,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-23 12:54:00,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-23 12:54:00,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-23 12:54:00,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-23 12:54:00,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-23 12:54:00,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-23 12:54:00,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-23 12:54:00,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-23 12:54:00,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-23 12:54:00,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-23 12:54:00,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-23 12:54:00,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-23 12:54:00,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-23 12:54:00,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-23 12:54:00,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-23 12:54:00,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-23 12:54:00,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-23 12:54:00,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-23 12:54:00,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-23 12:54:00,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-23 12:54:00,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-23 12:54:00,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-23 12:54:00,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-23 12:54:00,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-23 12:54:00,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-23 12:54:00,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-23 12:54:00,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-23 12:54:00,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-23 12:54:00,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-23 12:54:00,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-23 12:54:00,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-23 12:54:00,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-23 12:54:00,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-23 12:54:00,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-23 12:54:00,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-23 12:54:00,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-23 12:54:00,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-23 12:54:00,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-23 12:54:00,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-23 12:54:00,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-23 12:54:00,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-23 12:54:00,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-23 12:54:00,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-23 12:54:00,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-23 12:54:00,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-23 12:54:00,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-23 12:54:00,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-23 12:54:00,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-23 12:54:00,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-23 12:54:00,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-23 12:54:00,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-23 12:54:00,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-23 12:54:00,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-23 12:54:00,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-23 12:54:00,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-23 12:54:00,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-23 12:54:00,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-23 12:54:00,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-23 12:54:00,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-23 12:54:00,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-23 12:54:00,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-23 12:54:00,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-23 12:54:00,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-23 12:54:00,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-23 12:54:00,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-23 12:54:00,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-23 12:54:00,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-23 12:54:00,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-23 12:54:00,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-23 12:54:00,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-23 12:54:00,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-23 12:54:00,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-23 12:54:00,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-23 12:54:00,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-23 12:54:00,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-23 12:54:00,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-23 12:54:00,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-23 12:54:00,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-23 12:54:00,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-23 12:54:00,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-23 12:54:00,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-23 12:54:00,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-23 12:54:00,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-23 12:54:00,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-23 12:54:00,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-23 12:54:00,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-23 12:54:00,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-23 12:54:00,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-23 12:54:00,847 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03951255560111164, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05490732933602594, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03951255560111164, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17416666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24225, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-23 12:54:00,847 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-23 12:54:00,848 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17904480269165782, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03951255560111164, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05490732933602594, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03951255560111164, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17416666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24225, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-23 12:54:01,151 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 407 ms to try 5000 different iterations.  Found a solution that moves 2648 regions; Going from a computed imbalance of 0.17904480269165782 to a new imbalance of 0.01596893534930395. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020267570421104022); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.033424118794243736, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2942222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020267570421104022); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04525, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-23 12:54:01,162 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv127521909:77 , srv1938867924:77 , srv1044595535:80 , srv1706430659:81 , srv397517396:81 , srv1681369271:82 , srv1749357228:82 , srv1413638516:83 , srv1694361171:83 , srv2105747058:83 , srv542644028:83 , srv703713637:83 , srv813260134:83 , srv81633427:83 , srv1106956988:84 , srv1129778566:84 , srv1296874411:84 , srv1363114969:84 , srv1412033070:84 , srv15637876:84 , srv1941901387:84 , srv247801399:84 , srv290907532:84 , srv41972961:84 , srv66142673:84 , srv697554800:84 , srv770282871:84 , srv1254958702:85 , srv1298044022:85 , srv1320548417:85 , srv1385508202:85 , srv1484777926:85 , srv1605857888:85 , srv1714594973:85 , srv1768383670:85 , srv1940149584:85 , srv503015366:85 , srv516938821:85 , srv524358192:85 , srv622795080:85 , srv691043731:85 , srv743336876:85 , srv819325586:85 , srv823031647:85 , srv835665298:85 , srv1093633209:86 , srv1116767657:86 , srv1131885373:86 , srv1249829012:86 , srv1260379304:86 , srv1410262742:86 , srv159746081:86 , srv1717740466:86 , srv1780152375:86 , srv1807611876:86 , srv1951466482:86 , srv1981082403:86 , srv2115776532:86 , srv480794400:86 , srv481335518:86 , srv52492372:86 , srv53736856:86 , srv635654725:86 , srv698033253:86 , srv923086408:86 , srv971518830:86 , srv1068815799:87 , srv1172900549:87 , srv1201555398:87 , srv1282086934:87 , srv1424056366:87 , srv1730051979:87 , srv2047904159:87 , srv263529034:87 , srv354264732:87 , srv434402727:87 , srv663549327:87 , srv902934368:87 , srv112334816:88 , srv1554647824:88 , srv1554855276:88 , srv1646945984:88 , srv1708436565:88 , srv2060117409:88 , srv355414864:88 , srv950071303:88 , srv1559962570:89 , srv1929813241:89 , srv250121446:89 , srv571270375:89 , srv941739914:89 , srv1098629370:90 , srv1789907047:90 , srv1929506318:90 , srv1938400004:90 , srv784409722:90 , srv826455605:90 , srv1448136735:91 , srv2090829640:94 , srv1825796688:513 }
2023-02-23 12:54:01,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-23 12:54:01,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1385508202=20, srv1412033070=22, srv159746081=31, srv354264732=64, srv835665298=94, srv355414864=65, srv826455605=93, srv1044595535=0, srv112334816=6, srv1448136735=25, srv15637876=30, srv1131885373=8, srv697554800=83, srv971518830=99, srv1282086934=15, srv770282871=87, srv950071303=98, srv1825796688=46, srv902934368=95, srv1559962570=29, srv703713637=85, srv1938867924=50, srv1068815799=1, srv1941901387=52, srv1172900549=9, srv503015366=71, srv1260379304=13, srv1768383670=42, srv41972961=67, srv1929506318=47, srv52492372=74, srv1554647824=27, srv247801399=60, srv1413638516=23, srv2115776532=59, srv923086408=96, srv516938821=72, srv743336876=86, srv524358192=73, srv1681369271=34, srv1106956988=4, srv1780152375=43, srv263529034=62, srv290907532=63, srv1254958702=12, srv1981082403=54, srv819325586=91, srv1129778566=7, srv1708436565=37, srv1706430659=36, srv698033253=84, srv1484777926=26, srv1093633209=2, srv81633427=90, srv823031647=92, srv1951466482=53, srv1730051979=40, srv480794400=69, srv1807611876=45, srv1098629370=3, srv1363114969=19, srv1717740466=39, srv1714594973=38, srv663549327=81, srv941739914=97, srv1249829012=11, srv397517396=66, srv1940149584=51, srv1929813241=48, srv127521909=14, srv784409722=88, srv1694361171=35, srv2105747058=58, srv1749357228=41, srv571270375=77, srv2060117409=56, srv2090829640=57, srv1938400004=49, srv481335518=70, srv635654725=79, srv1298044022=17, srv1424056366=24, srv1554855276=28, srv622795080=78, srv1296874411=16, srv542644028=76, srv250121446=61, srv66142673=80, srv1605857888=32, srv53736856=75, srv2047904159=55, srv1320548417=18, srv434402727=68, srv813260134=89, srv1201555398=10, srv1789907047=44, srv1116767657=5, srv691043731=82, srv1410262742=21, srv1646945984=33} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-23 12:54:01,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-23 12:54:01,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-23 12:54:01,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-23 12:54:01,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-23 12:54:01,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-23 12:54:01,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-23 12:54:01,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-23 12:54:01,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-23 12:54:01,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-23 12:54:01,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-23 12:54:01,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-23 12:54:01,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-23 12:54:01,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-23 12:54:01,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-23 12:54:01,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-23 12:54:01,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-23 12:54:01,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-23 12:54:01,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-23 12:54:01,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-23 12:54:01,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-23 12:54:01,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-23 12:54:01,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-23 12:54:01,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-23 12:54:01,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-23 12:54:01,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-23 12:54:01,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-23 12:54:01,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-23 12:54:01,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-23 12:54:01,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-23 12:54:01,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-23 12:54:01,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-23 12:54:01,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-23 12:54:01,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-23 12:54:01,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-23 12:54:01,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-23 12:54:01,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-23 12:54:01,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-23 12:54:01,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-23 12:54:01,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-23 12:54:01,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-23 12:54:01,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-23 12:54:01,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-23 12:54:01,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-23 12:54:01,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-23 12:54:01,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-23 12:54:01,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-23 12:54:01,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-23 12:54:01,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-23 12:54:01,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-23 12:54:01,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-23 12:54:01,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-23 12:54:01,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-23 12:54:01,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-23 12:54:01,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-23 12:54:01,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-23 12:54:01,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-23 12:54:01,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-23 12:54:01,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-23 12:54:01,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-23 12:54:01,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-23 12:54:01,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-23 12:54:01,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-23 12:54:01,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-23 12:54:01,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-23 12:54:01,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-23 12:54:01,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-23 12:54:01,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-23 12:54:01,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-23 12:54:01,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-23 12:54:01,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-23 12:54:01,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-23 12:54:01,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-23 12:54:01,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-23 12:54:01,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-23 12:54:01,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-23 12:54:01,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-23 12:54:01,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-23 12:54:01,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-23 12:54:01,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-23 12:54:01,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-23 12:54:01,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-23 12:54:01,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-23 12:54:01,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-23 12:54:01,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-23 12:54:01,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-23 12:54:01,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-23 12:54:01,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-23 12:54:01,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-23 12:54:01,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-23 12:54:01,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-23 12:54:01,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-23 12:54:01,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-23 12:54:01,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-23 12:54:01,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-23 12:54:01,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-23 12:54:01,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-23 12:54:01,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-23 12:54:01,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-23 12:54:01,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-23 12:54:01,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-23 12:54:01,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-23 12:54:01,336 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020267570421104022); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.033424118794243736, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020267570421104022); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04525, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-23 12:54:01,337 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-23 12:54:01,337 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.015950393657004413, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020267570421104022); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.033424118794243736, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020267570421104022); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04525, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-23 12:54:01,627 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 402 ms to try 5000 different iterations.  Found a solution that moves 994 regions; Going from a computed imbalance of 0.015950393657004413 to a new imbalance of 5.231076232570334E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005204389640497334); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.010762198466155315); MoveCostFunction : (multiplier=7.0, imbalance=0.11044444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005204389640497334); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004083333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-23 12:54:01,629 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv784409722:81 , srv1938867924:82 , srv1201555398:83 , srv1424056366:83 , srv1559962570:83 , srv1681369271:83 , srv1789907047:83 , srv355414864:83 , srv524358192:83 , srv697554800:83 , srv743336876:83 , srv819325586:83 , srv1249829012:84 , srv1298044022:84 , srv1385508202:84 , srv1412033070:84 , srv1448136735:84 , srv1484777926:84 , srv15637876:84 , srv1646945984:84 , srv1717740466:84 , srv1929506318:84 , srv2090829640:84 , srv250121446:84 , srv354264732:84 , srv663549327:84 , srv770282871:84 , srv923086408:84 , srv941739914:84 , srv950071303:84 , srv1044595535:85 , srv112334816:85 , srv1131885373:85 , srv1172900549:85 , srv1320548417:85 , srv1363114969:85 , srv1768383670:85 , srv1929813241:85 , srv1951466482:85 , srv2115776532:85 , srv434402727:85 , srv53736856:85 , srv542644028:85 , srv622795080:85 , srv635654725:85 , srv691043731:85 , srv813260134:85 , srv971518830:85 , srv1068815799:86 , srv1093633209:86 , srv1098629370:86 , srv1254958702:86 , srv1260379304:86 , srv127521909:86 , srv1282086934:86 , srv1410262742:86 , srv1413638516:86 , srv1554647824:86 , srv1554855276:86 , srv1780152375:86 , srv1807611876:86 , srv1940149584:86 , srv1981082403:86 , srv2047904159:86 , srv2060117409:86 , srv247801399:86 , srv263529034:86 , srv397517396:86 , srv480794400:86 , srv516938821:86 , srv52492372:86 , srv571270375:86 , srv698033253:86 , srv81633427:86 , srv823031647:86 , srv826455605:86 , srv835665298:86 , srv902934368:86 , srv1106956988:87 , srv1129778566:87 , srv1694361171:87 , srv1708436565:87 , srv1730051979:87 , srv1749357228:87 , srv1938400004:87 , srv2105747058:87 , srv290907532:87 , srv41972961:87 , srv481335518:87 , srv503015366:87 , srv703713637:87 , srv1116767657:88 , srv1296874411:88 , srv159746081:88 , srv1605857888:88 , srv1706430659:88 , srv1714594973:88 , srv1941901387:88 , srv66142673:88 , srv1825796688:550 }
2023-02-23 12:54:01,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-23 12:54:01,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1385508202=20, srv1412033070=22, srv159746081=31, srv354264732=64, srv835665298=94, srv355414864=65, srv826455605=93, srv1044595535=0, srv112334816=6, srv1448136735=25, srv15637876=30, srv1131885373=8, srv697554800=83, srv971518830=99, srv1282086934=15, srv770282871=87, srv950071303=98, srv1825796688=46, srv902934368=95, srv1559962570=29, srv703713637=85, srv1938867924=50, srv1068815799=1, srv1941901387=52, srv1172900549=9, srv503015366=71, srv1260379304=13, srv1768383670=42, srv41972961=67, srv1929506318=47, srv52492372=74, srv1554647824=27, srv247801399=60, srv1413638516=23, srv2115776532=59, srv923086408=96, srv516938821=72, srv743336876=86, srv524358192=73, srv1681369271=34, srv1106956988=4, srv1780152375=43, srv263529034=62, srv290907532=63, srv1254958702=12, srv1981082403=54, srv819325586=91, srv1129778566=7, srv1708436565=37, srv1706430659=36, srv698033253=84, srv1484777926=26, srv1093633209=2, srv81633427=90, srv823031647=92, srv1951466482=53, srv1730051979=40, srv480794400=69, srv1807611876=45, srv1098629370=3, srv1363114969=19, srv1717740466=39, srv1714594973=38, srv663549327=81, srv941739914=97, srv1249829012=11, srv397517396=66, srv1940149584=51, srv1929813241=48, srv127521909=14, srv784409722=88, srv1694361171=35, srv2105747058=58, srv1749357228=41, srv571270375=77, srv2060117409=56, srv2090829640=57, srv1938400004=49, srv481335518=70, srv635654725=79, srv1298044022=17, srv1424056366=24, srv1554855276=28, srv622795080=78, srv1296874411=16, srv542644028=76, srv250121446=61, srv66142673=80, srv1605857888=32, srv53736856=75, srv2047904159=55, srv1320548417=18, srv434402727=68, srv813260134=89, srv1201555398=10, srv1789907047=44, srv1116767657=5, srv691043731=82, srv1410262742=21, srv1646945984=33} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-23 12:54:01,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-23 12:54:01,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-23 12:54:01,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-23 12:54:01,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-23 12:54:01,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-23 12:54:01,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-23 12:54:01,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-23 12:54:01,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-23 12:54:01,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-23 12:54:01,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-23 12:54:01,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-23 12:54:01,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-23 12:54:01,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-23 12:54:01,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-23 12:54:01,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-23 12:54:01,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-23 12:54:01,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-23 12:54:01,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-23 12:54:01,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-23 12:54:01,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-23 12:54:01,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-23 12:54:01,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-23 12:54:01,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-23 12:54:01,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-23 12:54:01,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-23 12:54:01,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-23 12:54:01,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-23 12:54:01,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-23 12:54:01,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-23 12:54:01,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-23 12:54:01,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-23 12:54:01,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-23 12:54:01,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-23 12:54:01,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-23 12:54:01,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-23 12:54:01,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-23 12:54:01,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-23 12:54:01,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-23 12:54:01,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-23 12:54:01,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-23 12:54:01,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-23 12:54:01,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-23 12:54:01,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-23 12:54:01,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-23 12:54:01,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-23 12:54:01,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-23 12:54:01,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-23 12:54:01,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-23 12:54:01,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-23 12:54:01,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-23 12:54:01,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-23 12:54:01,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-23 12:54:01,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-23 12:54:01,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-23 12:54:01,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-23 12:54:01,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-23 12:54:01,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-23 12:54:01,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-23 12:54:01,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-23 12:54:01,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-23 12:54:01,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-23 12:54:01,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-23 12:54:01,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-23 12:54:01,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-23 12:54:01,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-23 12:54:01,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-23 12:54:01,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-23 12:54:01,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-23 12:54:01,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-23 12:54:01,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-23 12:54:01,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-23 12:54:01,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-23 12:54:01,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-23 12:54:01,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-23 12:54:01,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-23 12:54:01,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-23 12:54:01,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-23 12:54:01,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-23 12:54:01,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-23 12:54:01,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-23 12:54:01,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-23 12:54:01,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-23 12:54:01,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-23 12:54:01,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-23 12:54:01,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-23 12:54:01,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-23 12:54:01,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-23 12:54:01,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-23 12:54:01,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-23 12:54:01,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-23 12:54:01,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-23 12:54:01,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-23 12:54:01,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-23 12:54:01,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-23 12:54:01,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-23 12:54:01,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-23 12:54:01,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-23 12:54:01,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-23 12:54:01,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-23 12:54:01,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-23 12:54:01,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-23 12:54:01,767 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005204389640497334); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.010762198466155315); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005204389640497334); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004083333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-23 12:54:01,767 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-23 12:54:01,767 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=5.161474864950475E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005204389640497334); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.010762198466155315); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005204389640497334); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004083333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-23 12:54:02,158 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 492 ms to try 5000 different iterations.  Found a solution that moves 262 regions; Going from a computed imbalance of 5.161474864950475E-4 to a new imbalance of 2.174491780590808E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.470732932414788E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0021711298242630987); MoveCostFunction : (multiplier=7.0, imbalance=0.029111111111111112, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.470732932414788E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-23 12:54:02,159 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1129778566:83 , srv1714594973:83 , srv1981082403:83 , srv481335518:83 , srv516938821:83 , srv66142673:83 , srv1068815799:84 , srv1098629370:84 , srv1131885373:84 , srv1201555398:84 , srv1260379304:84 , srv127521909:84 , srv1320548417:84 , srv1363114969:84 , srv1385508202:84 , srv1412033070:84 , srv1484777926:84 , srv1554647824:84 , srv1554855276:84 , srv15637876:84 , srv159746081:84 , srv1605857888:84 , srv1646945984:84 , srv1706430659:84 , srv1708436565:84 , srv1730051979:84 , srv1929506318:84 , srv1929813241:84 , srv1938867924:84 , srv1941901387:84 , srv2060117409:84 , srv2090829640:84 , srv2105747058:84 , srv2115776532:84 , srv247801399:84 , srv250121446:84 , srv263529034:84 , srv290907532:84 , srv354264732:84 , srv41972961:84 , srv434402727:84 , srv480794400:84 , srv503015366:84 , srv542644028:84 , srv691043731:84 , srv697554800:84 , srv698033253:84 , srv770282871:84 , srv81633427:84 , srv902934368:84 , srv950071303:84 , srv971518830:84 , srv1093633209:85 , srv1106956988:85 , srv1116767657:85 , srv112334816:85 , srv1172900549:85 , srv1249829012:85 , srv1254958702:85 , srv1282086934:85 , srv1296874411:85 , srv1410262742:85 , srv1413638516:85 , srv1424056366:85 , srv1448136735:85 , srv1559962570:85 , srv1681369271:85 , srv1694361171:85 , srv1717740466:85 , srv1749357228:85 , srv1768383670:85 , srv1780152375:85 , srv1807611876:85 , srv1938400004:85 , srv1940149584:85 , srv2047904159:85 , srv355414864:85 , srv397517396:85 , srv524358192:85 , srv52492372:85 , srv571270375:85 , srv622795080:85 , srv635654725:85 , srv663549327:85 , srv703713637:85 , srv743336876:85 , srv784409722:85 , srv813260134:85 , srv819325586:85 , srv826455605:85 , srv835665298:85 , srv923086408:85 , srv941739914:85 , srv1044595535:86 , srv1298044022:86 , srv1789907047:86 , srv1951466482:86 , srv53736856:86 , srv823031647:86 , srv1825796688:637 }
2023-02-23 12:54:02,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-23 12:54:02,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1385508202=20, srv1412033070=22, srv159746081=31, srv354264732=64, srv835665298=94, srv355414864=65, srv826455605=93, srv1044595535=0, srv112334816=6, srv1448136735=25, srv15637876=30, srv1131885373=8, srv697554800=83, srv971518830=99, srv1282086934=15, srv770282871=87, srv950071303=98, srv1825796688=46, srv902934368=95, srv1559962570=29, srv703713637=85, srv1938867924=50, srv1068815799=1, srv1941901387=52, srv1172900549=9, srv503015366=71, srv1260379304=13, srv1768383670=42, srv41972961=67, srv1929506318=47, srv52492372=74, srv1554647824=27, srv247801399=60, srv1413638516=23, srv2115776532=59, srv923086408=96, srv516938821=72, srv743336876=86, srv524358192=73, srv1681369271=34, srv1106956988=4, srv1780152375=43, srv263529034=62, srv290907532=63, srv1254958702=12, srv1981082403=54, srv819325586=91, srv1129778566=7, srv1708436565=37, srv1706430659=36, srv698033253=84, srv1484777926=26, srv1093633209=2, srv81633427=90, srv823031647=92, srv1951466482=53, srv1730051979=40, srv480794400=69, srv1807611876=45, srv1098629370=3, srv1363114969=19, srv1717740466=39, srv1714594973=38, srv663549327=81, srv941739914=97, srv1249829012=11, srv397517396=66, srv1940149584=51, srv1929813241=48, srv127521909=14, srv784409722=88, srv1694361171=35, srv2105747058=58, srv1749357228=41, srv571270375=77, srv2060117409=56, srv2090829640=57, srv1938400004=49, srv481335518=70, srv635654725=79, srv1298044022=17, srv1424056366=24, srv1554855276=28, srv622795080=78, srv1296874411=16, srv542644028=76, srv250121446=61, srv66142673=80, srv1605857888=32, srv53736856=75, srv2047904159=55, srv1320548417=18, srv434402727=68, srv813260134=89, srv1201555398=10, srv1789907047=44, srv1116767657=5, srv691043731=82, srv1410262742=21, srv1646945984=33} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-23 12:54:02,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-23 12:54:02,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-23 12:54:02,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-23 12:54:02,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-23 12:54:02,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-23 12:54:02,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-23 12:54:02,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-23 12:54:02,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-23 12:54:02,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-23 12:54:02,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-23 12:54:02,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-23 12:54:02,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-23 12:54:02,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-23 12:54:02,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-23 12:54:02,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-23 12:54:02,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-23 12:54:02,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-23 12:54:02,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-23 12:54:02,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-23 12:54:02,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-23 12:54:02,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-23 12:54:02,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-23 12:54:02,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-23 12:54:02,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-23 12:54:02,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-23 12:54:02,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-23 12:54:02,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-23 12:54:02,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-23 12:54:02,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-23 12:54:02,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-23 12:54:02,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-23 12:54:02,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-23 12:54:02,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-23 12:54:02,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-23 12:54:02,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-23 12:54:02,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-23 12:54:02,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-23 12:54:02,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-23 12:54:02,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-23 12:54:02,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-23 12:54:02,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-23 12:54:02,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-23 12:54:02,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-23 12:54:02,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-23 12:54:02,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-23 12:54:02,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-23 12:54:02,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-23 12:54:02,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-23 12:54:02,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-23 12:54:02,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-23 12:54:02,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-23 12:54:02,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-23 12:54:02,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-23 12:54:02,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-23 12:54:02,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-23 12:54:02,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-23 12:54:02,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-23 12:54:02,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-23 12:54:02,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-23 12:54:02,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-23 12:54:02,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-23 12:54:02,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-23 12:54:02,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-23 12:54:02,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-23 12:54:02,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-23 12:54:02,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-23 12:54:02,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-23 12:54:02,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-23 12:54:02,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-23 12:54:02,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-23 12:54:02,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-23 12:54:02,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-23 12:54:02,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-23 12:54:02,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-23 12:54:02,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-23 12:54:02,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-23 12:54:02,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-23 12:54:02,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-23 12:54:02,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-23 12:54:02,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-23 12:54:02,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-23 12:54:02,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-23 12:54:02,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-23 12:54:02,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-23 12:54:02,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-23 12:54:02,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-23 12:54:02,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-23 12:54:02,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-23 12:54:02,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-23 12:54:02,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-23 12:54:02,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-23 12:54:02,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-23 12:54:02,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-23 12:54:02,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-23 12:54:02,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-23 12:54:02,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-23 12:54:02,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-23 12:54:02,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-23 12:54:02,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-23 12:54:02,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-23 12:54:02,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-23 12:54:02,317 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.991035459500233E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.470732932414788E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0021711298242630987); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.470732932414788E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-23 12:54:02,318 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1129778566:83 , srv1714594973:83 , srv1981082403:83 , srv481335518:83 , srv516938821:83 , srv66142673:83 , srv1068815799:84 , srv1098629370:84 , srv1131885373:84 , srv1201555398:84 , srv1260379304:84 , srv127521909:84 , srv1320548417:84 , srv1363114969:84 , srv1385508202:84 , srv1412033070:84 , srv1484777926:84 , srv1554647824:84 , srv1554855276:84 , srv15637876:84 , srv159746081:84 , srv1605857888:84 , srv1646945984:84 , srv1706430659:84 , srv1708436565:84 , srv1730051979:84 , srv1929506318:84 , srv1929813241:84 , srv1938867924:84 , srv1941901387:84 , srv2060117409:84 , srv2090829640:84 , srv2105747058:84 , srv2115776532:84 , srv247801399:84 , srv250121446:84 , srv263529034:84 , srv290907532:84 , srv354264732:84 , srv41972961:84 , srv434402727:84 , srv480794400:84 , srv503015366:84 , srv542644028:84 , srv691043731:84 , srv697554800:84 , srv698033253:84 , srv770282871:84 , srv81633427:84 , srv902934368:84 , srv950071303:84 , srv971518830:84 , srv1093633209:85 , srv1106956988:85 , srv1116767657:85 , srv112334816:85 , srv1172900549:85 , srv1249829012:85 , srv1254958702:85 , srv1282086934:85 , srv1296874411:85 , srv1410262742:85 , srv1413638516:85 , srv1424056366:85 , srv1448136735:85 , srv1559962570:85 , srv1681369271:85 , srv1694361171:85 , srv1717740466:85 , srv1749357228:85 , srv1768383670:85 , srv1780152375:85 , srv1807611876:85 , srv1938400004:85 , srv1940149584:85 , srv2047904159:85 , srv355414864:85 , srv397517396:85 , srv524358192:85 , srv52492372:85 , srv571270375:85 , srv622795080:85 , srv635654725:85 , srv663549327:85 , srv703713637:85 , srv743336876:85 , srv784409722:85 , srv813260134:85 , srv819325586:85 , srv826455605:85 , srv835665298:85 , srv923086408:85 , srv941739914:85 , srv1044595535:86 , srv1298044022:86 , srv1789907047:86 , srv1951466482:86 , srv53736856:86 , srv823031647:86 , srv1825796688:637 }
2023-02-23 12:54:02,356 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=287 (was 286) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=6094 (was 6589)
2023-02-23 12:54:02,365 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=287, ProcessCount=170, AvailableMemoryMB=6094
2023-02-23 12:54:02,366 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-23 12:54:02,367 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-02-23 12:54:02,379 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1020535458:84 , srv1026819829:84 , srv1095142426:84 , srv1117580597:84 , srv1118965134:84 , srv1213931082:84 , srv1244325686:84 , srv1358025075:84 , srv1427095950:84 , srv1484980678:84 , srv1504504399:84 , srv1640452665:84 , srv1667082853:84 , srv1676055914:84 , srv1725268188:84 , srv1781140397:84 , srv1836816678:84 , srv1945049943:84 , srv2104831029:84 , srv220276470:84 , srv264377574:84 , srv304594066:84 , srv338054269:84 , srv405871865:84 , srv418818696:84 , srv486946612:84 , srv646875109:84 , srv725164738:84 , srv950907899:84 , srv1470450170:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-02-23 12:54:02,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-23 12:54:02,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv220276470=20, srv1213931082=5, srv338054269=23, srv1427095950=8, srv304594066=22, srv1118965134=4, srv1484980678=10, srv1781140397=16, srv950907899=29, srv1026819829=1, srv1640452665=12, srv1504504399=11, srv2104831029=19, srv418818696=25, srv1667082853=13, srv486946612=26, srv1244325686=6, srv1358025075=7, srv1836816678=17, srv1470450170=9, srv646875109=27, srv264377574=21, srv405871865=24, srv1725268188=15, srv1117580597=3, srv725164738=28, srv1020535458=0, srv1945049943=18, srv1676055914=14, srv1095142426=2} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-23 12:54:02,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-23 12:54:02,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-23 12:54:02,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-23 12:54:02,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-23 12:54:02,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-23 12:54:02,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-23 12:54:02,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-23 12:54:02,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-23 12:54:02,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-23 12:54:02,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-23 12:54:02,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-23 12:54:02,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-23 12:54:02,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-23 12:54:02,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-23 12:54:02,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-23 12:54:02,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-23 12:54:02,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-23 12:54:02,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-23 12:54:02,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-23 12:54:02,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-23 12:54:02,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-23 12:54:02,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-23 12:54:02,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-23 12:54:02,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-23 12:54:02,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-23 12:54:02,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-23 12:54:02,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-23 12:54:02,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-23 12:54:02,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-23 12:54:02,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-23 12:54:02,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-02-23 12:54:02,417 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1703694206477433, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-23 12:54:02,417 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912729871745666, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1703694206477433, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-02-23 12:56:02,400 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6138919 different iterations.  Found a solution that moves 1858 regions; Going from a computed imbalance of 0.9912729871745666 to a new imbalance of 4.336664689392977E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6881481481481482, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-23 12:56:02,407 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1020535458:90 , srv1026819829:90 , srv1095142426:90 , srv1117580597:90 , srv1118965134:90 , srv1213931082:90 , srv1244325686:90 , srv1358025075:90 , srv1427095950:90 , srv1470450170:90 , srv1484980678:90 , srv1504504399:90 , srv1640452665:90 , srv1667082853:90 , srv1676055914:90 , srv1725268188:90 , srv1781140397:90 , srv1836816678:90 , srv1945049943:90 , srv2104831029:90 , srv220276470:90 , srv264377574:90 , srv304594066:90 , srv338054269:90 , srv405871865:90 , srv418818696:90 , srv486946612:90 , srv646875109:90 , srv725164738:90 , srv950907899:90 }
2023-02-23 12:56:02,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-23 12:56:02,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv220276470=20, srv1213931082=5, srv338054269=23, srv1427095950=8, srv304594066=22, srv1118965134=4, srv1484980678=10, srv1781140397=16, srv950907899=29, srv1026819829=1, srv1640452665=12, srv1504504399=11, srv2104831029=19, srv418818696=25, srv1667082853=13, srv486946612=26, srv1244325686=6, srv1358025075=7, srv1836816678=17, srv1470450170=9, srv646875109=27, srv264377574=21, srv405871865=24, srv1725268188=15, srv1117580597=3, srv725164738=28, srv1020535458=0, srv1945049943=18, srv1676055914=14, srv1095142426=2} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-23 12:56:02,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-23 12:56:02,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-23 12:56:02,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-23 12:56:02,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-23 12:56:02,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-23 12:56:02,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-23 12:56:02,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-23 12:56:02,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-23 12:56:02,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-23 12:56:02,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-23 12:56:02,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-23 12:56:02,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-23 12:56:02,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-23 12:56:02,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-23 12:56:02,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-23 12:56:02,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-23 12:56:02,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-23 12:56:02,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-23 12:56:02,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-23 12:56:02,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-23 12:56:02,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-23 12:56:02,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-23 12:56:02,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-23 12:56:02,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-23 12:56:02,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-23 12:56:02,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-23 12:56:02,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-23 12:56:02,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-23 12:56:02,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-23 12:56:02,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-23 12:56:02,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-02-23 12:56:02,437 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-23 12:56:02,437 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1020535458:90 , srv1026819829:90 , srv1095142426:90 , srv1117580597:90 , srv1118965134:90 , srv1213931082:90 , srv1244325686:90 , srv1358025075:90 , srv1427095950:90 , srv1470450170:90 , srv1484980678:90 , srv1504504399:90 , srv1640452665:90 , srv1667082853:90 , srv1676055914:90 , srv1725268188:90 , srv1781140397:90 , srv1836816678:90 , srv1945049943:90 , srv2104831029:90 , srv220276470:90 , srv264377574:90 , srv304594066:90 , srv338054269:90 , srv405871865:90 , srv418818696:90 , srv486946612:90 , srv646875109:90 , srv725164738:90 , srv950907899:90 }
2023-02-23 12:56:02,462 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=213 (was 287), ProcessCount=170 (was 170), AvailableMemoryMB=6034 (was 6094)