2023-02-22 02:54:03,548 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-02-22 02:54:03,861 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-02-22 02:54:03,990 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-02-22 02:54:04,050 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-22 02:54:04,057 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-02-22 02:54:04,071 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=290, ProcessCount=172, AvailableMemoryMB=8286
2023-02-22 02:54:04,073 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-22 02:54:04,073 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-02-22 02:54:04,214 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv102273478:84 , srv1043051801:84 , srv1047636822:84 , srv1095189319:84 , srv1127083217:84 , srv1152710994:84 , srv1218557628:84 , srv1252848342:84 , srv1256835531:84 , srv1261125853:84 , srv1287774786:84 , srv1337581549:84 , srv1338542349:84 , srv1341388118:84 , srv1345056093:84 , srv1436220578:84 , srv1437814393:84 , srv1458721492:84 , srv1463908686:84 , srv1472825740:84 , srv1484037839:84 , srv1486375766:84 , srv1512618103:84 , srv1519475558:84 , srv1521693901:84 , srv1538588023:84 , srv1550196671:84 , srv1553365113:84 , srv1560479557:84 , srv1581720156:84 , srv1584990984:84 , srv1590071497:84 , srv1598529963:84 , srv1611590205:84 , srv1644709468:84 , srv1653334591:84 , srv165589163:84 , srv1671642286:84 , srv1675860637:84 , srv1682846433:84 , srv169258359:84 , srv1710441180:84 , srv171285677:84 , srv1714213656:84 , srv171708932:84 , srv1742059237:84 , srv1762906672:84 , srv1807468531:84 , srv1825860875:84 , srv1837377932:84 , srv1881714006:84 , srv1920642715:84 , srv1928527815:84 , srv1932408321:84 , srv1935319574:84 , srv1938352475:84 , srv20366950:84 , srv2070401535:84 , srv2092346368:84 , srv2104955544:84 , srv2117283404:84 , srv2119726490:84 , srv2122718511:84 , srv216901947:84 , srv238361258:84 , srv258208055:84 , srv276659651:84 , srv297808840:84 , srv320911956:84 , srv395209574:84 , srv405452685:84 , srv412325290:84 , srv44635723:84 , srv453656728:84 , srv460886349:84 , srv465375920:84 , srv486469558:84 , srv493491238:84 , srv543452241:84 , srv562992498:84 , srv593205221:84 , srv613904787:84 , srv654546798:84 , srv660667667:84 , srv667324595:84 , srv671402375:84 , srv685329071:84 , srv715090981:84 , srv773421728:84 , srv804026386:84 , srv814677062:84 , srv825632580:84 , srv832008538:84 , srv846128533:84 , srv861977040:84 , srv962481199:84 , srv966415073:84 , srv970526546:84 , srv972112322:84 , srv1442922981:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-02-22 02:54:04,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-22 02:54:04,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1252848342=7, srv2122718511=63, srv814677062=91, srv2070401535=58, srv1095189319=3, srv1837377932=50, srv238361258=65, srv671402375=86, srv20366950=57, srv1611590205=34, srv1338542349=12, srv1437814393=16, srv543452241=79, srv1519475558=24, srv1261125853=9, srv1584990984=31, srv1807468531=48, srv1538588023=26, srv593205221=81, srv660667667=84, srv1553365113=28, srv1675860637=39, srv1512618103=23, srv1152710994=5, srv1287774786=10, srv216901947=64, srv1550196671=27, srv1043051801=1, srv2092346368=59, srv453656728=74, srv970526546=98, srv1486375766=22, srv171708932=45, srv1598529963=33, srv465375920=76, srv1928527815=53, srv1442922981=17, srv102273478=0, srv1714213656=44, srv493491238=78, srv773421728=89, srv2117283404=61, srv972112322=99, srv171285677=43, srv1560479557=29, srv1653334591=36, srv562992498=80, srv412325290=72, srv654546798=83, srv1337581549=11, srv276659651=67, srv1521693901=25, srv169258359=41, srv1436220578=15, srv1932408321=54, srv1458721492=18, srv395209574=70, srv832008538=93, srv1881714006=51, srv804026386=90, srv1345056093=14, srv1472825740=20, srv715090981=88, srv460886349=75, srv667324595=85, srv44635723=73, srv1935319574=55, srv2104955544=60, srv405452685=71, srv861977040=95, srv1644709468=35, srv2119726490=62, srv1127083217=4, srv962481199=96, srv258208055=66, srv966415073=97, srv1256835531=8, srv1590071497=32, srv1341388118=13, srv1710441180=42, srv1484037839=21, srv685329071=87, srv1762906672=47, srv1671642286=38, srv1742059237=46, srv486469558=77, srv846128533=94, srv1463908686=19, srv1047636822=2, srv1825860875=49, srv1920642715=52, srv1938352475=56, srv297808840=68, srv1682846433=40, srv320911956=69, srv1581720156=30, srv165589163=37, srv825632580=92, srv613904787=82, srv1218557628=6} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-22 02:54:04,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-22 02:54:04,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-22 02:54:04,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-22 02:54:04,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-22 02:54:04,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-22 02:54:04,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-22 02:54:04,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-22 02:54:04,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-22 02:54:04,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-22 02:54:04,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-22 02:54:04,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-22 02:54:04,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-22 02:54:04,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-22 02:54:04,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-22 02:54:04,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-22 02:54:04,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-22 02:54:04,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-22 02:54:04,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-22 02:54:04,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-22 02:54:04,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-22 02:54:04,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-22 02:54:04,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-22 02:54:04,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-22 02:54:04,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-22 02:54:04,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-22 02:54:04,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-22 02:54:04,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-22 02:54:04,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-22 02:54:04,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-22 02:54:04,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-22 02:54:04,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-22 02:54:04,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-22 02:54:04,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-22 02:54:04,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-22 02:54:04,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-22 02:54:04,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-22 02:54:04,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-22 02:54:04,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-22 02:54:04,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-22 02:54:04,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-22 02:54:04,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-22 02:54:04,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-22 02:54:04,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-22 02:54:04,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-22 02:54:04,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-22 02:54:04,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-22 02:54:04,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-22 02:54:04,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-22 02:54:04,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-22 02:54:04,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-22 02:54:04,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-22 02:54:04,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-22 02:54:04,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-22 02:54:04,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-22 02:54:04,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-22 02:54:04,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-22 02:54:04,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-22 02:54:04,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-22 02:54:04,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-22 02:54:04,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-22 02:54:04,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-22 02:54:04,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-22 02:54:04,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-22 02:54:04,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-22 02:54:04,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-22 02:54:04,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-22 02:54:04,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-22 02:54:04,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-22 02:54:04,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-22 02:54:04,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-22 02:54:04,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-22 02:54:04,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-22 02:54:04,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-22 02:54:04,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-22 02:54:04,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-22 02:54:04,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-22 02:54:04,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-22 02:54:04,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-22 02:54:04,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-22 02:54:04,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-22 02:54:04,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-22 02:54:04,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-22 02:54:04,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-22 02:54:04,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-22 02:54:04,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-22 02:54:04,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-22 02:54:04,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-22 02:54:04,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-22 02:54:04,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-22 02:54:04,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-22 02:54:04,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-22 02:54:04,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-22 02:54:04,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-22 02:54:04,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-22 02:54:04,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-22 02:54:04,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-22 02:54:04,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-22 02:54:04,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-22 02:54:04,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-22 02:54:04,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-22 02:54:04,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-22 02:54:04,540 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-22 02:54:04,541 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-22 02:54:04,541 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-22 02:54:04,852 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 486 ms to try 5000 different iterations.  Found a solution that moves 4684 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18099111001628948. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03928593211192755, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05357688497678768, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5204444444444445, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03928593211192755, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1765, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24025, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-22 02:54:04,874 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1436220578:82 , srv486469558:82 , srv966415073:82 , srv1338542349:83 , srv1345056093:83 , srv1484037839:83 , srv1553365113:83 , srv1644709468:83 , srv1710441180:83 , srv1762906672:83 , srv2104955544:83 , srv562992498:83 , srv1047636822:84 , srv1152710994:84 , srv1252848342:84 , srv1256835531:84 , srv1337581549:84 , srv1486375766:84 , srv1581720156:84 , srv1671642286:84 , srv1682846433:84 , srv2122718511:84 , srv258208055:84 , srv405452685:84 , srv804026386:84 , srv814677062:84 , srv1560479557:85 , srv1584990984:85 , srv1611590205:85 , srv1742059237:85 , srv1928527815:85 , srv1935319574:85 , srv2092346368:85 , srv216901947:85 , srv460886349:85 , srv543452241:85 , srv593205221:85 , srv861977040:85 , srv962481199:85 , srv1218557628:86 , srv1521693901:86 , srv1550196671:86 , srv1598529963:86 , srv165589163:86 , srv1675860637:86 , srv1714213656:86 , srv171708932:86 , srv1825860875:86 , srv1881714006:86 , srv1932408321:86 , srv465375920:86 , srv613904787:86 , srv654546798:86 , srv660667667:86 , srv715090981:86 , srv1043051801:87 , srv1095189319:87 , srv1127083217:87 , srv1463908686:87 , srv1590071497:87 , srv1837377932:87 , srv1938352475:87 , srv2070401535:87 , srv2119726490:87 , srv276659651:87 , srv320911956:87 , srv493491238:87 , srv685329071:87 , srv102273478:88 , srv1261125853:88 , srv1341388118:88 , srv1472825740:88 , srv1512618103:88 , srv1920642715:88 , srv2117283404:88 , srv395209574:88 , srv773421728:88 , srv825632580:88 , srv832008538:88 , srv846128533:88 , srv1437814393:89 , srv1458721492:89 , srv1519475558:89 , srv171285677:89 , srv20366950:89 , srv297808840:89 , srv671402375:89 , srv972112322:89 , srv1653334591:90 , srv1807468531:90 , srv1287774786:91 , srv238361258:91 , srv453656728:91 , srv412325290:92 , srv44635723:92 , srv970526546:92 , srv1538588023:94 , srv169258359:94 , srv667324595:95 , srv1442922981:439 }
2023-02-22 02:54:04,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-22 02:54:04,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1252848342=7, srv2122718511=63, srv814677062=91, srv2070401535=58, srv1095189319=3, srv1837377932=50, srv238361258=65, srv671402375=86, srv20366950=57, srv1611590205=34, srv1338542349=12, srv1437814393=16, srv543452241=79, srv1519475558=24, srv1261125853=9, srv1584990984=31, srv1807468531=48, srv1538588023=26, srv593205221=81, srv660667667=84, srv1553365113=28, srv1675860637=39, srv1512618103=23, srv1152710994=5, srv1287774786=10, srv216901947=64, srv1550196671=27, srv1043051801=1, srv2092346368=59, srv453656728=74, srv970526546=98, srv1486375766=22, srv171708932=45, srv1598529963=33, srv465375920=76, srv1928527815=53, srv1442922981=17, srv102273478=0, srv1714213656=44, srv493491238=78, srv773421728=89, srv2117283404=61, srv972112322=99, srv171285677=43, srv1560479557=29, srv1653334591=36, srv562992498=80, srv412325290=72, srv654546798=83, srv1337581549=11, srv276659651=67, srv1521693901=25, srv169258359=41, srv1436220578=15, srv1932408321=54, srv1458721492=18, srv395209574=70, srv832008538=93, srv1881714006=51, srv804026386=90, srv1345056093=14, srv1472825740=20, srv715090981=88, srv460886349=75, srv667324595=85, srv44635723=73, srv1935319574=55, srv2104955544=60, srv405452685=71, srv861977040=95, srv1644709468=35, srv2119726490=62, srv1127083217=4, srv962481199=96, srv258208055=66, srv966415073=97, srv1256835531=8, srv1590071497=32, srv1341388118=13, srv1710441180=42, srv1484037839=21, srv685329071=87, srv1762906672=47, srv1671642286=38, srv1742059237=46, srv486469558=77, srv846128533=94, srv1463908686=19, srv1047636822=2, srv1825860875=49, srv1920642715=52, srv1938352475=56, srv297808840=68, srv1682846433=40, srv320911956=69, srv1581720156=30, srv165589163=37, srv825632580=92, srv613904787=82, srv1218557628=6} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-22 02:54:04,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-22 02:54:04,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-22 02:54:04,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-22 02:54:04,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-22 02:54:04,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-22 02:54:04,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-22 02:54:04,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-22 02:54:04,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-22 02:54:04,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-22 02:54:04,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-22 02:54:04,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-22 02:54:04,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-22 02:54:04,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-22 02:54:04,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-22 02:54:04,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-22 02:54:04,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-22 02:54:04,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-22 02:54:04,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-22 02:54:04,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-22 02:54:04,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-22 02:54:04,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-22 02:54:04,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-22 02:54:04,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-22 02:54:04,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-22 02:54:04,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-22 02:54:04,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-22 02:54:04,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-22 02:54:04,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-22 02:54:04,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-22 02:54:04,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-22 02:54:04,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-22 02:54:04,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-22 02:54:04,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-22 02:54:04,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-22 02:54:04,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-22 02:54:04,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-22 02:54:04,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-22 02:54:04,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-22 02:54:04,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-22 02:54:04,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-22 02:54:04,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-22 02:54:04,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-22 02:54:04,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-22 02:54:04,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-22 02:54:04,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-22 02:54:04,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-22 02:54:04,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-22 02:54:04,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-22 02:54:04,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-22 02:54:04,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-22 02:54:04,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-22 02:54:04,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-22 02:54:04,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-22 02:54:04,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-22 02:54:04,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-22 02:54:04,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-22 02:54:04,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-22 02:54:04,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-22 02:54:04,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-22 02:54:04,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-22 02:54:04,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-22 02:54:04,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-22 02:54:04,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-22 02:54:04,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-22 02:54:04,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-22 02:54:04,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-22 02:54:04,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-22 02:54:04,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-22 02:54:04,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-22 02:54:04,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-22 02:54:04,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-22 02:54:04,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-22 02:54:04,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-22 02:54:04,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-22 02:54:04,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-22 02:54:04,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-22 02:54:04,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-22 02:54:04,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-22 02:54:04,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-22 02:54:04,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-22 02:54:04,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-22 02:54:04,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-22 02:54:04,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-22 02:54:04,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-22 02:54:04,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-22 02:54:04,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-22 02:54:04,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-22 02:54:04,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-22 02:54:04,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-22 02:54:04,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-22 02:54:04,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-22 02:54:04,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-22 02:54:04,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-22 02:54:04,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-22 02:54:04,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-22 02:54:04,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-22 02:54:04,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-22 02:54:04,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-22 02:54:04,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-22 02:54:04,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-22 02:54:04,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-22 02:54:05,020 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03928593211192755, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05357688497678768, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03928593211192755, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1765, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24025, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-22 02:54:05,021 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-22 02:54:05,021 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.1809583119472823, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03928593211192755, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05357688497678768, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03928593211192755, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1765, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24025, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-22 02:54:05,276 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 357 ms to try 5000 different iterations.  Found a solution that moves 2680 regions; Going from a computed imbalance of 0.1809583119472823 to a new imbalance of 0.012877810322045161. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020372512896202774); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.034889596595794624, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29777777777777775, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020372512896202774); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.009666666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043333333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-22 02:54:05,286 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv667324595:76 , srv169258359:77 , srv1538588023:79 , srv44635723:79 , srv1287774786:80 , srv238361258:81 , srv970526546:81 , srv1472825740:82 , srv1512618103:82 , srv1807468531:82 , srv20366950:82 , srv412325290:82 , srv453656728:82 , srv671402375:82 , srv102273478:83 , srv1458721492:83 , srv1653334591:83 , srv1920642715:83 , srv2119726490:83 , srv297808840:83 , srv1127083217:84 , srv1261125853:84 , srv1341388118:84 , srv1437814393:84 , srv1521693901:84 , srv171285677:84 , srv2092346368:84 , srv2117283404:84 , srv465375920:84 , srv832008538:84 , srv846128533:84 , srv972112322:84 , srv1463908686:85 , srv1590071497:85 , srv1598529963:85 , srv1611590205:85 , srv165589163:85 , srv1825860875:85 , srv1837377932:85 , srv1938352475:85 , srv2070401535:85 , srv276659651:85 , srv395209574:85 , srv493491238:85 , srv715090981:85 , srv1095189319:86 , srv1252848342:86 , srv1519475558:86 , srv1584990984:86 , srv1675860637:86 , srv171708932:86 , srv1932408321:86 , srv320911956:86 , srv460886349:86 , srv593205221:86 , srv685329071:86 , srv861977040:86 , srv1043051801:87 , srv1152710994:87 , srv1218557628:87 , srv1256835531:87 , srv1345056093:87 , srv1682846433:87 , srv1762906672:87 , srv1881714006:87 , srv543452241:87 , srv660667667:87 , srv773421728:87 , srv825632580:87 , srv962481199:87 , srv1484037839:88 , srv1486375766:88 , srv1550196671:88 , srv1560479557:88 , srv1581720156:88 , srv1710441180:88 , srv1742059237:88 , srv216901947:88 , srv258208055:88 , srv405452685:88 , srv1047636822:89 , srv1553365113:89 , srv1644709468:89 , srv1714213656:89 , srv1935319574:89 , srv2104955544:89 , srv2122718511:89 , srv562992498:89 , srv654546798:89 , srv804026386:89 , srv814677062:89 , srv1671642286:90 , srv1928527815:90 , srv486469558:90 , srv613904787:90 , srv1337581549:91 , srv1338542349:91 , srv1436220578:91 , srv966415073:91 , srv1442922981:516 }
2023-02-22 02:54:05,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-22 02:54:05,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1252848342=7, srv2122718511=63, srv814677062=91, srv2070401535=58, srv1095189319=3, srv1837377932=50, srv238361258=65, srv671402375=86, srv20366950=57, srv1611590205=34, srv1338542349=12, srv1437814393=16, srv543452241=79, srv1519475558=24, srv1261125853=9, srv1584990984=31, srv1807468531=48, srv1538588023=26, srv593205221=81, srv660667667=84, srv1553365113=28, srv1675860637=39, srv1512618103=23, srv1152710994=5, srv1287774786=10, srv216901947=64, srv1550196671=27, srv1043051801=1, srv2092346368=59, srv453656728=74, srv970526546=98, srv1486375766=22, srv171708932=45, srv1598529963=33, srv465375920=76, srv1928527815=53, srv1442922981=17, srv102273478=0, srv1714213656=44, srv493491238=78, srv773421728=89, srv2117283404=61, srv972112322=99, srv171285677=43, srv1560479557=29, srv1653334591=36, srv562992498=80, srv412325290=72, srv654546798=83, srv1337581549=11, srv276659651=67, srv1521693901=25, srv169258359=41, srv1436220578=15, srv1932408321=54, srv1458721492=18, srv395209574=70, srv832008538=93, srv1881714006=51, srv804026386=90, srv1345056093=14, srv1472825740=20, srv715090981=88, srv460886349=75, srv667324595=85, srv44635723=73, srv1935319574=55, srv2104955544=60, srv405452685=71, srv861977040=95, srv1644709468=35, srv2119726490=62, srv1127083217=4, srv962481199=96, srv258208055=66, srv966415073=97, srv1256835531=8, srv1590071497=32, srv1341388118=13, srv1710441180=42, srv1484037839=21, srv685329071=87, srv1762906672=47, srv1671642286=38, srv1742059237=46, srv486469558=77, srv846128533=94, srv1463908686=19, srv1047636822=2, srv1825860875=49, srv1920642715=52, srv1938352475=56, srv297808840=68, srv1682846433=40, srv320911956=69, srv1581720156=30, srv165589163=37, srv825632580=92, srv613904787=82, srv1218557628=6} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-22 02:54:05,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-22 02:54:05,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-22 02:54:05,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-22 02:54:05,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-22 02:54:05,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-22 02:54:05,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-22 02:54:05,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-22 02:54:05,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-22 02:54:05,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-22 02:54:05,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-22 02:54:05,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-22 02:54:05,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-22 02:54:05,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-22 02:54:05,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-22 02:54:05,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-22 02:54:05,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-22 02:54:05,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-22 02:54:05,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-22 02:54:05,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-22 02:54:05,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-22 02:54:05,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-22 02:54:05,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-22 02:54:05,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-22 02:54:05,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-22 02:54:05,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-22 02:54:05,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-22 02:54:05,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-22 02:54:05,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-22 02:54:05,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-22 02:54:05,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-22 02:54:05,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-22 02:54:05,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-22 02:54:05,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-22 02:54:05,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-22 02:54:05,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-22 02:54:05,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-22 02:54:05,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-22 02:54:05,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-22 02:54:05,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-22 02:54:05,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-22 02:54:05,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-22 02:54:05,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-22 02:54:05,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-22 02:54:05,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-22 02:54:05,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-22 02:54:05,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-22 02:54:05,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-22 02:54:05,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-22 02:54:05,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-22 02:54:05,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-22 02:54:05,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-22 02:54:05,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-22 02:54:05,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-22 02:54:05,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-22 02:54:05,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-22 02:54:05,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-22 02:54:05,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-22 02:54:05,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-22 02:54:05,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-22 02:54:05,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-22 02:54:05,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-22 02:54:05,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-22 02:54:05,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-22 02:54:05,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-22 02:54:05,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-22 02:54:05,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-22 02:54:05,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-22 02:54:05,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-22 02:54:05,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-22 02:54:05,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-22 02:54:05,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-22 02:54:05,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-22 02:54:05,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-22 02:54:05,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-22 02:54:05,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-22 02:54:05,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-22 02:54:05,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-22 02:54:05,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-22 02:54:05,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-22 02:54:05,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-22 02:54:05,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-22 02:54:05,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-22 02:54:05,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-22 02:54:05,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-22 02:54:05,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-22 02:54:05,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-22 02:54:05,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-22 02:54:05,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-22 02:54:05,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-22 02:54:05,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-22 02:54:05,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-22 02:54:05,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-22 02:54:05,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-22 02:54:05,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-22 02:54:05,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-22 02:54:05,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-22 02:54:05,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-22 02:54:05,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-22 02:54:05,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-22 02:54:05,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-22 02:54:05,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-22 02:54:05,486 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020372512896202774); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.034889596595794624, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020372512896202774); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.009666666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043333333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-22 02:54:05,486 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-22 02:54:05,486 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.012859044560956505, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020372512896202774); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.034889596595794624, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020372512896202774); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.009666666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043333333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-22 02:54:05,738 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 397 ms to try 5000 different iterations.  Found a solution that moves 996 regions; Going from a computed imbalance of 0.012859044560956505 to a new imbalance of 0.0010301575594398337. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006330861746253887); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008863600391305173); MoveCostFunction : (multiplier=7.0, imbalance=0.11066666666666666, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006330861746253887); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=6.666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003916666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-22 02:54:05,740 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv613904787:81 , srv1043051801:82 , srv1928527815:82 , srv1337581549:83 , srv1519475558:83 , srv1714213656:83 , srv654546798:83 , srv773421728:83 , srv825632580:83 , srv1047636822:84 , srv1218557628:84 , srv1338542349:84 , srv1436220578:84 , srv1538588023:84 , srv1550196671:84 , srv1590071497:84 , srv1671642286:84 , srv1742059237:84 , srv1881714006:84 , srv1932408321:84 , srv1935319574:84 , srv2117283404:84 , srv216901947:84 , srv320911956:84 , srv395209574:84 , srv412325290:84 , srv486469558:84 , srv685329071:84 , srv804026386:84 , srv970526546:84 , srv1095189319:85 , srv1261125853:85 , srv1341388118:85 , srv1458721492:85 , srv1486375766:85 , srv1553365113:85 , srv1560479557:85 , srv1581720156:85 , srv1653334591:85 , srv171285677:85 , srv1807468531:85 , srv2070401535:85 , srv2104955544:85 , srv2122718511:85 , srv276659651:85 , srv453656728:85 , srv493491238:85 , srv660667667:85 , srv667324595:85 , srv832008538:85 , srv846128533:85 , srv966415073:85 , srv972112322:85 , srv102273478:86 , srv1127083217:86 , srv1256835531:86 , srv1287774786:86 , srv1345056093:86 , srv1437814393:86 , srv1463908686:86 , srv1584990984:86 , srv1598529963:86 , srv1644709468:86 , srv165589163:86 , srv1682846433:86 , srv169258359:86 , srv1710441180:86 , srv171708932:86 , srv1825860875:86 , srv1837377932:86 , srv1938352475:86 , srv238361258:86 , srv258208055:86 , srv297808840:86 , srv405452685:86 , srv44635723:86 , srv543452241:86 , srv562992498:86 , srv671402375:86 , srv814677062:86 , srv861977040:86 , srv962481199:86 , srv1152710994:87 , srv1472825740:87 , srv1512618103:87 , srv1521693901:87 , srv1611590205:87 , srv1675860637:87 , srv1920642715:87 , srv20366950:87 , srv2119726490:87 , srv460886349:87 , srv593205221:87 , srv715090981:87 , srv1252848342:88 , srv1484037839:88 , srv1762906672:88 , srv465375920:88 , srv2092346368:89 , srv1442922981:559 }
2023-02-22 02:54:05,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-22 02:54:05,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1252848342=7, srv2122718511=63, srv814677062=91, srv2070401535=58, srv1095189319=3, srv1837377932=50, srv238361258=65, srv671402375=86, srv20366950=57, srv1611590205=34, srv1338542349=12, srv1437814393=16, srv543452241=79, srv1519475558=24, srv1261125853=9, srv1584990984=31, srv1807468531=48, srv1538588023=26, srv593205221=81, srv660667667=84, srv1553365113=28, srv1675860637=39, srv1512618103=23, srv1152710994=5, srv1287774786=10, srv216901947=64, srv1550196671=27, srv1043051801=1, srv2092346368=59, srv453656728=74, srv970526546=98, srv1486375766=22, srv171708932=45, srv1598529963=33, srv465375920=76, srv1928527815=53, srv1442922981=17, srv102273478=0, srv1714213656=44, srv493491238=78, srv773421728=89, srv2117283404=61, srv972112322=99, srv171285677=43, srv1560479557=29, srv1653334591=36, srv562992498=80, srv412325290=72, srv654546798=83, srv1337581549=11, srv276659651=67, srv1521693901=25, srv169258359=41, srv1436220578=15, srv1932408321=54, srv1458721492=18, srv395209574=70, srv832008538=93, srv1881714006=51, srv804026386=90, srv1345056093=14, srv1472825740=20, srv715090981=88, srv460886349=75, srv667324595=85, srv44635723=73, srv1935319574=55, srv2104955544=60, srv405452685=71, srv861977040=95, srv1644709468=35, srv2119726490=62, srv1127083217=4, srv962481199=96, srv258208055=66, srv966415073=97, srv1256835531=8, srv1590071497=32, srv1341388118=13, srv1710441180=42, srv1484037839=21, srv685329071=87, srv1762906672=47, srv1671642286=38, srv1742059237=46, srv486469558=77, srv846128533=94, srv1463908686=19, srv1047636822=2, srv1825860875=49, srv1920642715=52, srv1938352475=56, srv297808840=68, srv1682846433=40, srv320911956=69, srv1581720156=30, srv165589163=37, srv825632580=92, srv613904787=82, srv1218557628=6} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-22 02:54:05,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-22 02:54:05,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-22 02:54:05,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-22 02:54:05,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-22 02:54:05,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-22 02:54:05,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-22 02:54:05,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-22 02:54:05,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-22 02:54:05,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-22 02:54:05,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-22 02:54:05,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-22 02:54:05,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-22 02:54:05,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-22 02:54:05,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-22 02:54:05,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-22 02:54:05,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-22 02:54:05,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-22 02:54:05,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-22 02:54:05,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-22 02:54:05,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-22 02:54:05,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-22 02:54:05,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-22 02:54:05,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-22 02:54:05,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-22 02:54:05,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-22 02:54:05,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-22 02:54:05,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-22 02:54:05,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-22 02:54:05,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-22 02:54:05,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-22 02:54:05,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-22 02:54:05,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-22 02:54:05,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-22 02:54:05,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-22 02:54:05,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-22 02:54:05,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-22 02:54:05,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-22 02:54:05,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-22 02:54:05,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-22 02:54:05,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-22 02:54:05,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-22 02:54:05,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-22 02:54:05,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-22 02:54:05,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-22 02:54:05,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-22 02:54:05,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-22 02:54:05,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-22 02:54:05,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-22 02:54:05,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-22 02:54:05,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-22 02:54:05,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-22 02:54:05,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-22 02:54:05,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-22 02:54:05,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-22 02:54:05,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-22 02:54:05,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-22 02:54:05,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-22 02:54:05,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-22 02:54:05,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-22 02:54:05,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-22 02:54:05,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-22 02:54:05,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-22 02:54:05,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-22 02:54:05,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-22 02:54:05,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-22 02:54:05,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-22 02:54:05,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-22 02:54:05,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-22 02:54:05,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-22 02:54:05,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-22 02:54:05,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-22 02:54:05,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-22 02:54:05,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-22 02:54:05,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-22 02:54:05,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-22 02:54:05,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-22 02:54:05,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-22 02:54:05,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-22 02:54:05,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-22 02:54:05,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-22 02:54:05,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-22 02:54:05,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-22 02:54:05,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-22 02:54:05,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-22 02:54:05,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-22 02:54:05,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-22 02:54:05,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-22 02:54:05,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-22 02:54:05,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-22 02:54:05,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-22 02:54:05,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-22 02:54:05,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-22 02:54:05,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-22 02:54:05,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-22 02:54:05,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-22 02:54:05,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-22 02:54:05,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-22 02:54:05,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-22 02:54:05,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-22 02:54:05,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-22 02:54:05,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-22 02:54:05,874 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006330861746253887); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008863600391305173); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006330861746253887); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=6.666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003916666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-22 02:54:05,875 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-22 02:54:05,875 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.001023183418378528, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006330861746253887); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008863600391305173); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006330861746253887); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=6.666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003916666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-22 02:54:06,218 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 440 ms to try 5000 different iterations.  Found a solution that moves 259 regions; Going from a computed imbalance of 0.001023183418378528 to a new imbalance of 4.3393676382214265E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.23782800878924E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.028777777777777777, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.23782800878924E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-22 02:54:06,219 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1437814393:83 , srv1484037839:83 , srv1598529963:83 , srv1675860637:83 , srv171708932:83 , srv814677062:83 , srv1095189319:84 , srv1152710994:84 , srv1252848342:84 , srv1338542349:84 , srv1341388118:84 , srv1463908686:84 , srv1521693901:84 , srv1550196671:84 , srv1560479557:84 , srv1581720156:84 , srv1644709468:84 , srv1653334591:84 , srv1671642286:84 , srv169258359:84 , srv171285677:84 , srv1762906672:84 , srv1837377932:84 , srv1920642715:84 , srv1928527815:84 , srv1935319574:84 , srv20366950:84 , srv2092346368:84 , srv2122718511:84 , srv238361258:84 , srv258208055:84 , srv297808840:84 , srv412325290:84 , srv453656728:84 , srv460886349:84 , srv493491238:84 , srv543452241:84 , srv562992498:84 , srv593205221:84 , srv654546798:84 , srv660667667:84 , srv715090981:84 , srv773421728:84 , srv962481199:84 , srv966415073:84 , srv972112322:84 , srv102273478:85 , srv1047636822:85 , srv1127083217:85 , srv1218557628:85 , srv1256835531:85 , srv1261125853:85 , srv1287774786:85 , srv1337581549:85 , srv1436220578:85 , srv1458721492:85 , srv1486375766:85 , srv1512618103:85 , srv1519475558:85 , srv1538588023:85 , srv1553365113:85 , srv1611590205:85 , srv165589163:85 , srv1682846433:85 , srv1710441180:85 , srv1714213656:85 , srv1742059237:85 , srv1807468531:85 , srv1825860875:85 , srv1881714006:85 , srv1938352475:85 , srv2070401535:85 , srv2104955544:85 , srv2119726490:85 , srv216901947:85 , srv276659651:85 , srv320911956:85 , srv395209574:85 , srv405452685:85 , srv465375920:85 , srv613904787:85 , srv667324595:85 , srv671402375:85 , srv685329071:85 , srv804026386:85 , srv825632580:85 , srv832008538:85 , srv846128533:85 , srv861977040:85 , srv970526546:85 , srv1043051801:86 , srv1345056093:86 , srv1472825740:86 , srv1584990984:86 , srv1590071497:86 , srv1932408321:86 , srv2117283404:86 , srv44635723:86 , srv486469558:86 , srv1442922981:628 }
2023-02-22 02:54:06,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-22 02:54:06,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1252848342=7, srv2122718511=63, srv814677062=91, srv2070401535=58, srv1095189319=3, srv1837377932=50, srv238361258=65, srv671402375=86, srv20366950=57, srv1611590205=34, srv1338542349=12, srv1437814393=16, srv543452241=79, srv1519475558=24, srv1261125853=9, srv1584990984=31, srv1807468531=48, srv1538588023=26, srv593205221=81, srv660667667=84, srv1553365113=28, srv1675860637=39, srv1512618103=23, srv1152710994=5, srv1287774786=10, srv216901947=64, srv1550196671=27, srv1043051801=1, srv2092346368=59, srv453656728=74, srv970526546=98, srv1486375766=22, srv171708932=45, srv1598529963=33, srv465375920=76, srv1928527815=53, srv1442922981=17, srv102273478=0, srv1714213656=44, srv493491238=78, srv773421728=89, srv2117283404=61, srv972112322=99, srv171285677=43, srv1560479557=29, srv1653334591=36, srv562992498=80, srv412325290=72, srv654546798=83, srv1337581549=11, srv276659651=67, srv1521693901=25, srv169258359=41, srv1436220578=15, srv1932408321=54, srv1458721492=18, srv395209574=70, srv832008538=93, srv1881714006=51, srv804026386=90, srv1345056093=14, srv1472825740=20, srv715090981=88, srv460886349=75, srv667324595=85, srv44635723=73, srv1935319574=55, srv2104955544=60, srv405452685=71, srv861977040=95, srv1644709468=35, srv2119726490=62, srv1127083217=4, srv962481199=96, srv258208055=66, srv966415073=97, srv1256835531=8, srv1590071497=32, srv1341388118=13, srv1710441180=42, srv1484037839=21, srv685329071=87, srv1762906672=47, srv1671642286=38, srv1742059237=46, srv486469558=77, srv846128533=94, srv1463908686=19, srv1047636822=2, srv1825860875=49, srv1920642715=52, srv1938352475=56, srv297808840=68, srv1682846433=40, srv320911956=69, srv1581720156=30, srv165589163=37, srv825632580=92, srv613904787=82, srv1218557628=6} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-22 02:54:06,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-22 02:54:06,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-22 02:54:06,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-22 02:54:06,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-22 02:54:06,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-22 02:54:06,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-22 02:54:06,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-22 02:54:06,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-22 02:54:06,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-22 02:54:06,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-22 02:54:06,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-22 02:54:06,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-22 02:54:06,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-22 02:54:06,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-22 02:54:06,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-22 02:54:06,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-22 02:54:06,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-22 02:54:06,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-22 02:54:06,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-22 02:54:06,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-22 02:54:06,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-22 02:54:06,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-22 02:54:06,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-22 02:54:06,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-22 02:54:06,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-22 02:54:06,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-22 02:54:06,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-22 02:54:06,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-22 02:54:06,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-22 02:54:06,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-22 02:54:06,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-22 02:54:06,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-22 02:54:06,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-22 02:54:06,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-22 02:54:06,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-22 02:54:06,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-22 02:54:06,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-22 02:54:06,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-22 02:54:06,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-22 02:54:06,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-22 02:54:06,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-22 02:54:06,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-22 02:54:06,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-22 02:54:06,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-22 02:54:06,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-22 02:54:06,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-22 02:54:06,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-22 02:54:06,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-22 02:54:06,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-22 02:54:06,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-22 02:54:06,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-22 02:54:06,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-22 02:54:06,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-22 02:54:06,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-22 02:54:06,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-22 02:54:06,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-22 02:54:06,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-22 02:54:06,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-22 02:54:06,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-22 02:54:06,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-22 02:54:06,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-22 02:54:06,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-22 02:54:06,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-22 02:54:06,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-22 02:54:06,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-22 02:54:06,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-22 02:54:06,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-22 02:54:06,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-22 02:54:06,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-22 02:54:06,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-22 02:54:06,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-22 02:54:06,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-22 02:54:06,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-22 02:54:06,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-22 02:54:06,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-22 02:54:06,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-22 02:54:06,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-22 02:54:06,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-22 02:54:06,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-22 02:54:06,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-22 02:54:06,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-22 02:54:06,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-22 02:54:06,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-22 02:54:06,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-22 02:54:06,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-22 02:54:06,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-22 02:54:06,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-22 02:54:06,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-22 02:54:06,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-22 02:54:06,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-22 02:54:06,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-22 02:54:06,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-22 02:54:06,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-22 02:54:06,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-22 02:54:06,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-22 02:54:06,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-22 02:54:06,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-22 02:54:06,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-22 02:54:06,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-22 02:54:06,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-22 02:54:06,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-22 02:54:06,345 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.23782800878924E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.23782800878924E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-22 02:54:06,345 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-22 02:54:06,345 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=4.158011962028835E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.23782800878924E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.23782800878924E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-22 02:54:06,595 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 346 ms to try 5000 different iterations.  Found a solution that moves 26 regions; Going from a computed imbalance of 4.158011962028835E-5 to a new imbalance of 8.161746008199175E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017727200782610348); MoveCostFunction : (multiplier=7.0, imbalance=0.0028888888888888888); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-22 02:54:06,595 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1337581549:83 , srv1472825740:83 , srv1519475558:83 , srv1584990984:83 , srv1714213656:83 , srv1938352475:83 , srv405452685:83 , srv44635723:83 , srv465375920:83 , srv825632580:83 , srv102273478:84 , srv1043051801:84 , srv1047636822:84 , srv1095189319:84 , srv1127083217:84 , srv1152710994:84 , srv1218557628:84 , srv1252848342:84 , srv1256835531:84 , srv1261125853:84 , srv1287774786:84 , srv1338542349:84 , srv1345056093:84 , srv1436220578:84 , srv1437814393:84 , srv1458721492:84 , srv1463908686:84 , srv1484037839:84 , srv1486375766:84 , srv1512618103:84 , srv1538588023:84 , srv1550196671:84 , srv1553365113:84 , srv1560479557:84 , srv1590071497:84 , srv1611590205:84 , srv1644709468:84 , srv1653334591:84 , srv165589163:84 , srv1671642286:84 , srv1675860637:84 , srv1682846433:84 , srv1710441180:84 , srv171285677:84 , srv1742059237:84 , srv1762906672:84 , srv1807468531:84 , srv1825860875:84 , srv1837377932:84 , srv1881714006:84 , srv1920642715:84 , srv1932408321:84 , srv1935319574:84 , srv20366950:84 , srv2070401535:84 , srv2092346368:84 , srv2104955544:84 , srv2117283404:84 , srv2119726490:84 , srv2122718511:84 , srv216901947:84 , srv238361258:84 , srv258208055:84 , srv276659651:84 , srv297808840:84 , srv320911956:84 , srv395209574:84 , srv412325290:84 , srv453656728:84 , srv460886349:84 , srv486469558:84 , srv543452241:84 , srv562992498:84 , srv593205221:84 , srv613904787:84 , srv654546798:84 , srv660667667:84 , srv671402375:84 , srv685329071:84 , srv715090981:84 , srv773421728:84 , srv804026386:84 , srv814677062:84 , srv832008538:84 , srv846128533:84 , srv861977040:84 , srv962481199:84 , srv966415073:84 , srv970526546:84 , srv972112322:84 , srv1341388118:85 , srv1521693901:85 , srv1581720156:85 , srv169258359:85 , srv171708932:85 , srv1928527815:85 , srv493491238:85 , srv667324595:85 , srv1598529963:86 , srv1442922981:684 }
2023-02-22 02:54:06,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-22 02:54:06,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1252848342=7, srv2122718511=63, srv814677062=91, srv2070401535=58, srv1095189319=3, srv1837377932=50, srv238361258=65, srv671402375=86, srv20366950=57, srv1611590205=34, srv1338542349=12, srv1437814393=16, srv543452241=79, srv1519475558=24, srv1261125853=9, srv1584990984=31, srv1807468531=48, srv1538588023=26, srv593205221=81, srv660667667=84, srv1553365113=28, srv1675860637=39, srv1512618103=23, srv1152710994=5, srv1287774786=10, srv216901947=64, srv1550196671=27, srv1043051801=1, srv2092346368=59, srv453656728=74, srv970526546=98, srv1486375766=22, srv171708932=45, srv1598529963=33, srv465375920=76, srv1928527815=53, srv1442922981=17, srv102273478=0, srv1714213656=44, srv493491238=78, srv773421728=89, srv2117283404=61, srv972112322=99, srv171285677=43, srv1560479557=29, srv1653334591=36, srv562992498=80, srv412325290=72, srv654546798=83, srv1337581549=11, srv276659651=67, srv1521693901=25, srv169258359=41, srv1436220578=15, srv1932408321=54, srv1458721492=18, srv395209574=70, srv832008538=93, srv1881714006=51, srv804026386=90, srv1345056093=14, srv1472825740=20, srv715090981=88, srv460886349=75, srv667324595=85, srv44635723=73, srv1935319574=55, srv2104955544=60, srv405452685=71, srv861977040=95, srv1644709468=35, srv2119726490=62, srv1127083217=4, srv962481199=96, srv258208055=66, srv966415073=97, srv1256835531=8, srv1590071497=32, srv1341388118=13, srv1710441180=42, srv1484037839=21, srv685329071=87, srv1762906672=47, srv1671642286=38, srv1742059237=46, srv486469558=77, srv846128533=94, srv1463908686=19, srv1047636822=2, srv1825860875=49, srv1920642715=52, srv1938352475=56, srv297808840=68, srv1682846433=40, srv320911956=69, srv1581720156=30, srv165589163=37, srv825632580=92, srv613904787=82, srv1218557628=6} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-22 02:54:06,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-22 02:54:06,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-22 02:54:06,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-22 02:54:06,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-22 02:54:06,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-22 02:54:06,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-22 02:54:06,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-22 02:54:06,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-22 02:54:06,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-22 02:54:06,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-22 02:54:06,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-22 02:54:06,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-22 02:54:06,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-22 02:54:06,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-22 02:54:06,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-22 02:54:06,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-22 02:54:06,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-22 02:54:06,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-22 02:54:06,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-22 02:54:06,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-22 02:54:06,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-22 02:54:06,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-22 02:54:06,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-22 02:54:06,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-22 02:54:06,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-22 02:54:06,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-22 02:54:06,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-22 02:54:06,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-22 02:54:06,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-22 02:54:06,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-22 02:54:06,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-22 02:54:06,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-22 02:54:06,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-22 02:54:06,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-22 02:54:06,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-22 02:54:06,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-22 02:54:06,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-22 02:54:06,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-22 02:54:06,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-22 02:54:06,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-22 02:54:06,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-22 02:54:06,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-22 02:54:06,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-22 02:54:06,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-22 02:54:06,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-22 02:54:06,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-22 02:54:06,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-22 02:54:06,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-22 02:54:06,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-22 02:54:06,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-22 02:54:06,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-22 02:54:06,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-22 02:54:06,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-22 02:54:06,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-22 02:54:06,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-22 02:54:06,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-22 02:54:06,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-22 02:54:06,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-22 02:54:06,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-22 02:54:06,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-22 02:54:06,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-22 02:54:06,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-22 02:54:06,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-22 02:54:06,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-22 02:54:06,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-22 02:54:06,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-22 02:54:06,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-22 02:54:06,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-22 02:54:06,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-22 02:54:06,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-22 02:54:06,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-22 02:54:06,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-22 02:54:06,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-22 02:54:06,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-22 02:54:06,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-22 02:54:06,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-22 02:54:06,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-22 02:54:06,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-22 02:54:06,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-22 02:54:06,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-22 02:54:06,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-22 02:54:06,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-22 02:54:06,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-22 02:54:06,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-22 02:54:06,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-22 02:54:06,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-22 02:54:06,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-22 02:54:06,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-22 02:54:06,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-22 02:54:06,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-22 02:54:06,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-22 02:54:06,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-22 02:54:06,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-22 02:54:06,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-22 02:54:06,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-22 02:54:06,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-22 02:54:06,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-22 02:54:06,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-22 02:54:06,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-22 02:54:06,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-22 02:54:06,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-22 02:54:06,735 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=7.979690117040589E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017727200782610348); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-22 02:54:06,735 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1337581549:83 , srv1472825740:83 , srv1519475558:83 , srv1584990984:83 , srv1714213656:83 , srv1938352475:83 , srv405452685:83 , srv44635723:83 , srv465375920:83 , srv825632580:83 , srv102273478:84 , srv1043051801:84 , srv1047636822:84 , srv1095189319:84 , srv1127083217:84 , srv1152710994:84 , srv1218557628:84 , srv1252848342:84 , srv1256835531:84 , srv1261125853:84 , srv1287774786:84 , srv1338542349:84 , srv1345056093:84 , srv1436220578:84 , srv1437814393:84 , srv1458721492:84 , srv1463908686:84 , srv1484037839:84 , srv1486375766:84 , srv1512618103:84 , srv1538588023:84 , srv1550196671:84 , srv1553365113:84 , srv1560479557:84 , srv1590071497:84 , srv1611590205:84 , srv1644709468:84 , srv1653334591:84 , srv165589163:84 , srv1671642286:84 , srv1675860637:84 , srv1682846433:84 , srv1710441180:84 , srv171285677:84 , srv1742059237:84 , srv1762906672:84 , srv1807468531:84 , srv1825860875:84 , srv1837377932:84 , srv1881714006:84 , srv1920642715:84 , srv1932408321:84 , srv1935319574:84 , srv20366950:84 , srv2070401535:84 , srv2092346368:84 , srv2104955544:84 , srv2117283404:84 , srv2119726490:84 , srv2122718511:84 , srv216901947:84 , srv238361258:84 , srv258208055:84 , srv276659651:84 , srv297808840:84 , srv320911956:84 , srv395209574:84 , srv412325290:84 , srv453656728:84 , srv460886349:84 , srv486469558:84 , srv543452241:84 , srv562992498:84 , srv593205221:84 , srv613904787:84 , srv654546798:84 , srv660667667:84 , srv671402375:84 , srv685329071:84 , srv715090981:84 , srv773421728:84 , srv804026386:84 , srv814677062:84 , srv832008538:84 , srv846128533:84 , srv861977040:84 , srv962481199:84 , srv966415073:84 , srv970526546:84 , srv972112322:84 , srv1341388118:85 , srv1521693901:85 , srv1581720156:85 , srv169258359:85 , srv171708932:85 , srv1928527815:85 , srv493491238:85 , srv667324595:85 , srv1598529963:86 , srv1442922981:684 }
2023-02-22 02:54:06,759 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=290 (was 290), ProcessCount=172 (was 172), AvailableMemoryMB=7774 (was 8286)
2023-02-22 02:54:06,767 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=290, ProcessCount=172, AvailableMemoryMB=7773
2023-02-22 02:54:06,768 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-22 02:54:06,768 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-02-22 02:54:06,779 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1014168106:84 , srv1097008362:84 , srv1121682385:84 , srv1181298593:84 , srv126980028:84 , srv129305892:84 , srv1355176003:84 , srv1443488945:84 , srv14520130:84 , srv1636248224:84 , srv1913892140:84 , srv1982000296:84 , srv2076617684:84 , srv2118261469:84 , srv212457229:84 , srv257136922:84 , srv404039480:84 , srv490658128:84 , srv549033659:84 , srv550489022:84 , srv569577045:84 , srv669336807:84 , srv779976429:84 , srv846028670:84 , srv871255729:84 , srv885029313:84 , srv96099652:84 , srv972453739:84 , srv982110907:84 , srv1529337705:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-02-22 02:54:06,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-22 02:54:06,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1181298593=3, srv1443488945=7, srv96099652=27, srv404039480=17, srv1355176003=6, srv550489022=20, srv1014168106=0, srv126980028=4, srv1913892140=11, srv549033659=19, srv257136922=16, srv982110907=29, srv2076617684=13, srv871255729=25, srv972453739=28, srv779976429=23, srv129305892=5, srv569577045=21, srv1636248224=10, srv2118261469=14, srv669336807=22, srv490658128=18, srv1097008362=1, srv1529337705=9, srv212457229=15, srv1982000296=12, srv14520130=8, srv846028670=24, srv885029313=26, srv1121682385=2} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-22 02:54:06,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-22 02:54:06,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-22 02:54:06,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-22 02:54:06,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-22 02:54:06,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-22 02:54:06,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-22 02:54:06,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-22 02:54:06,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-22 02:54:06,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-22 02:54:06,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-22 02:54:06,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-22 02:54:06,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-22 02:54:06,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-22 02:54:06,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-22 02:54:06,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-22 02:54:06,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-22 02:54:06,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-22 02:54:06,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-22 02:54:06,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-22 02:54:06,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-22 02:54:06,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-22 02:54:06,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-22 02:54:06,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-22 02:54:06,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-22 02:54:06,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-22 02:54:06,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-22 02:54:06,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-22 02:54:06,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-22 02:54:06,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-22 02:54:06,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-22 02:54:06,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-02-22 02:54:06,813 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0898243738238096, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-22 02:54:06,814 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912476076933164, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0898243738238096, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-02-22 02:56:06,799 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6194009 different iterations.  Found a solution that moves 1854 regions; Going from a computed imbalance of 0.9912476076933164 to a new imbalance of 4.3273284898463826E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6866666666666666, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-22 02:56:06,806 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1014168106:90 , srv1097008362:90 , srv1121682385:90 , srv1181298593:90 , srv126980028:90 , srv129305892:90 , srv1355176003:90 , srv1443488945:90 , srv14520130:90 , srv1529337705:90 , srv1636248224:90 , srv1913892140:90 , srv1982000296:90 , srv2076617684:90 , srv2118261469:90 , srv212457229:90 , srv257136922:90 , srv404039480:90 , srv490658128:90 , srv549033659:90 , srv550489022:90 , srv569577045:90 , srv669336807:90 , srv779976429:90 , srv846028670:90 , srv871255729:90 , srv885029313:90 , srv96099652:90 , srv972453739:90 , srv982110907:90 }
2023-02-22 02:56:06,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-22 02:56:06,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1181298593=3, srv1443488945=7, srv96099652=27, srv404039480=17, srv1355176003=6, srv550489022=20, srv1014168106=0, srv126980028=4, srv1913892140=11, srv549033659=19, srv257136922=16, srv982110907=29, srv2076617684=13, srv871255729=25, srv972453739=28, srv779976429=23, srv129305892=5, srv569577045=21, srv1636248224=10, srv2118261469=14, srv669336807=22, srv490658128=18, srv1097008362=1, srv1529337705=9, srv212457229=15, srv1982000296=12, srv14520130=8, srv846028670=24, srv885029313=26, srv1121682385=2} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-22 02:56:06,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-22 02:56:06,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-22 02:56:06,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-22 02:56:06,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-22 02:56:06,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-22 02:56:06,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-22 02:56:06,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-22 02:56:06,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-22 02:56:06,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-22 02:56:06,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-22 02:56:06,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-22 02:56:06,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-22 02:56:06,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-22 02:56:06,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-22 02:56:06,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-22 02:56:06,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-22 02:56:06,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-22 02:56:06,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-22 02:56:06,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-22 02:56:06,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-22 02:56:06,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-22 02:56:06,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-22 02:56:06,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-22 02:56:06,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-22 02:56:06,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-22 02:56:06,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-22 02:56:06,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-22 02:56:06,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-22 02:56:06,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-22 02:56:06,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-22 02:56:06,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-02-22 02:56:06,829 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-22 02:56:06,829 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1014168106:90 , srv1097008362:90 , srv1121682385:90 , srv1181298593:90 , srv126980028:90 , srv129305892:90 , srv1355176003:90 , srv1443488945:90 , srv14520130:90 , srv1529337705:90 , srv1636248224:90 , srv1913892140:90 , srv1982000296:90 , srv2076617684:90 , srv2118261469:90 , srv212457229:90 , srv257136922:90 , srv404039480:90 , srv490658128:90 , srv549033659:90 , srv550489022:90 , srv569577045:90 , srv669336807:90 , srv779976429:90 , srv846028670:90 , srv871255729:90 , srv885029313:90 , srv96099652:90 , srv972453739:90 , srv982110907:90 }
2023-02-22 02:56:06,842 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=202 (was 290), ProcessCount=172 (was 172), AvailableMemoryMB=7553 (was 7773)