2023-02-12 03:59:20,471 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-02-12 03:59:20,750 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-02-12 03:59:20,872 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-02-12 03:59:20,924 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-12 03:59:20,931 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-02-12 03:59:20,942 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=271, ProcessCount=164, AvailableMemoryMB=4574
2023-02-12 03:59:20,944 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-12 03:59:20,944 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-02-12 03:59:21,051 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1025477327:84 , srv1038192386:84 , srv107845026:84 , srv1080565524:84 , srv1096981633:84 , srv1157300237:84 , srv1162557737:84 , srv1169703964:84 , srv1170295766:84 , srv1170431799:84 , srv1170545250:84 , srv1178147673:84 , srv1183694488:84 , srv1205537780:84 , srv1244194825:84 , srv1250185792:84 , srv1250434833:84 , srv125352517:84 , srv1253940433:84 , srv1317275019:84 , srv1320822633:84 , srv1329163961:84 , srv136131977:84 , srv1396146779:84 , srv1415136137:84 , srv1416447314:84 , srv1429114152:84 , srv1432234805:84 , srv1480549808:84 , srv1498756220:84 , srv1512894696:84 , srv1521223098:84 , srv1538584483:84 , srv1553602073:84 , srv155685183:84 , srv1565431045:84 , srv1575470588:84 , srv1631347951:84 , srv1635759831:84 , srv1673579370:84 , srv1674725761:84 , srv1737141428:84 , srv1781525340:84 , srv1796519974:84 , srv182168835:84 , srv185365900:84 , srv1882814044:84 , srv1888934153:84 , srv1936206570:84 , srv1937703779:84 , srv1994490565:84 , srv1996300:84 , srv2016347424:84 , srv2019187920:84 , srv2073699485:84 , srv2104069446:84 , srv2110637285:84 , srv225980568:84 , srv249354630:84 , srv265771931:84 , srv272847005:84 , srv298823962:84 , srv304311585:84 , srv306552623:84 , srv349599513:84 , srv402531003:84 , srv404031598:84 , srv418118198:84 , srv450032989:84 , srv454090579:84 , srv457465505:84 , srv460533288:84 , srv476372959:84 , srv482432863:84 , srv486828065:84 , srv49332837:84 , srv498751518:84 , srv552601984:84 , srv564996557:84 , srv588701304:84 , srv613247825:84 , srv613745491:84 , srv622180497:84 , srv673123909:84 , srv674898333:84 , srv69705026:84 , srv743622037:84 , srv747986406:84 , srv763342768:84 , srv771693962:84 , srv798597569:84 , srv798830557:84 , srv874297738:84 , srv906466163:84 , srv916322977:84 , srv921448887:84 , srv930658942:84 , srv948625601:84 , srv989124097:84 , srv299007220:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-02-12 03:59:21,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-12 03:59:21,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1575470588=36, srv1521223098=31, srv1737141428=41, srv1674725761=40, srv948625601=98, srv349599513=65, srv1320822633=20, srv1205537780=13, srv182168835=44, srv155685183=34, srv673123909=84, srv989124097=99, srv1512894696=30, srv457465505=71, srv107845026=2, srv1169703964=7, srv1553602073=33, srv476372959=73, srv185365900=45, srv1796519974=43, srv249354630=58, srv454090579=70, srv1994490565=50, srv763342768=89, srv298823962=61, srv482432863=74, srv1329163961=21, srv771693962=90, srv2019187920=53, srv1157300237=5, srv1415136137=24, srv1888934153=47, srv874297738=93, srv1080565524=3, srv125352517=17, srv304311585=63, srv402531003=66, srv564996557=79, srv1038192386=1, srv299007220=62, srv450032989=69, srv1162557737=6, srv622180497=83, srv1170431799=9, srv674898333=85, srv916322977=95, srv1781525340=42, srv2110637285=56, srv1416447314=25, srv2104069446=55, srv306552623=64, srv1635759831=38, srv747986406=88, srv613745491=82, srv1498756220=29, srv69705026=86, srv798597569=91, srv1429114152=26, srv1432234805=27, srv1250434833=16, srv2016347424=52, srv49332837=76, srv906466163=94, srv1882814044=46, srv1936206570=48, srv225980568=57, srv1025477327=0, srv418118198=68, srv1170295766=8, srv588701304=80, srv798830557=92, srv613247825=81, srv1178147673=11, srv930658942=97, srv1317275019=19, srv1631347951=37, srv404031598=67, srv921448887=96, srv2073699485=54, srv1565431045=35, srv1183694488=12, srv136131977=22, srv1996300=51, srv1396146779=23, srv498751518=77, srv272847005=60, srv1538584483=32, srv486828065=75, srv1244194825=14, srv460533288=72, srv1253940433=18, srv1480549808=28, srv1937703779=49, srv1250185792=15, srv743622037=87, srv552601984=78, srv1170545250=10, srv1673579370=39, srv1096981633=4, srv265771931=59} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-12 03:59:21,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-12 03:59:21,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-12 03:59:21,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-12 03:59:21,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-12 03:59:21,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-12 03:59:21,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-12 03:59:21,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-12 03:59:21,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-12 03:59:21,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-12 03:59:21,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-12 03:59:21,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-12 03:59:21,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-12 03:59:21,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-12 03:59:21,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-12 03:59:21,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-12 03:59:21,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-12 03:59:21,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-12 03:59:21,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-12 03:59:21,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-12 03:59:21,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-12 03:59:21,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-12 03:59:21,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-12 03:59:21,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-12 03:59:21,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-12 03:59:21,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-12 03:59:21,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-12 03:59:21,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-12 03:59:21,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-12 03:59:21,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-12 03:59:21,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-12 03:59:21,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-12 03:59:21,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-12 03:59:21,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-12 03:59:21,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-12 03:59:21,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-12 03:59:21,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-12 03:59:21,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-12 03:59:21,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-12 03:59:21,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-12 03:59:21,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-12 03:59:21,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-12 03:59:21,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-12 03:59:21,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-12 03:59:21,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-12 03:59:21,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-12 03:59:21,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-12 03:59:21,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-12 03:59:21,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-12 03:59:21,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-12 03:59:21,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-12 03:59:21,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-12 03:59:21,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-12 03:59:21,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-12 03:59:21,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-12 03:59:21,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-12 03:59:21,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-12 03:59:21,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-12 03:59:21,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-12 03:59:21,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-12 03:59:21,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-12 03:59:21,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-12 03:59:21,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-12 03:59:21,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-12 03:59:21,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-12 03:59:21,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-12 03:59:21,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-12 03:59:21,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-12 03:59:21,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-12 03:59:21,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-12 03:59:21,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-12 03:59:21,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-12 03:59:21,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-12 03:59:21,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-12 03:59:21,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-12 03:59:21,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-12 03:59:21,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-12 03:59:21,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-12 03:59:21,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-12 03:59:21,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-12 03:59:21,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-12 03:59:21,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-12 03:59:21,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-12 03:59:21,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-12 03:59:21,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-12 03:59:21,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-12 03:59:21,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-12 03:59:21,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-12 03:59:21,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-12 03:59:21,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-12 03:59:21,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-12 03:59:21,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-12 03:59:21,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-12 03:59:21,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-12 03:59:21,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-12 03:59:21,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-12 03:59:21,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-12 03:59:21,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-12 03:59:21,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-12 03:59:21,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-12 03:59:21,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-12 03:59:21,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-12 03:59:21,292 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-12 03:59:21,292 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-12 03:59:21,292 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-12 03:59:21,513 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 325 ms to try 5000 different iterations.  Found a solution that moves 4742 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1719537424599482. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040511753169985724, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0561543487776701, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5268888888888889, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040511753169985724, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.167, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23466666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-12 03:59:21,522 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv2110637285:80 , srv1317275019:82 , srv107845026:83 , srv1157300237:83 , srv1320822633:83 , srv1329163961:83 , srv1415136137:83 , srv1538584483:83 , srv418118198:83 , srv457465505:83 , srv916322977:83 , srv1480549808:84 , srv1498756220:84 , srv1565431045:84 , srv1888934153:84 , srv2019187920:84 , srv2073699485:84 , srv298823962:84 , srv349599513:84 , srv404031598:84 , srv450032989:84 , srv588701304:84 , srv763342768:84 , srv771693962:84 , srv948625601:84 , srv989124097:84 , srv1038192386:85 , srv1170431799:85 , srv1521223098:85 , srv185365900:85 , srv1882814044:85 , srv1937703779:85 , srv304311585:85 , srv402531003:85 , srv674898333:85 , srv743622037:85 , srv921448887:85 , srv930658942:85 , srv1025477327:86 , srv1170545250:86 , srv1178147673:86 , srv1205537780:86 , srv125352517:86 , srv1396146779:86 , srv1432234805:86 , srv155685183:86 , srv1631347951:86 , srv1737141428:86 , srv272847005:86 , srv460533288:86 , srv476372959:86 , srv49332837:86 , srv498751518:86 , srv552601984:86 , srv564996557:86 , srv622180497:86 , srv798597569:86 , srv874297738:86 , srv1170295766:87 , srv1250185792:87 , srv136131977:87 , srv1553602073:87 , srv1674725761:87 , srv1996300:87 , srv2016347424:87 , srv2104069446:87 , srv265771931:87 , srv454090579:87 , srv798830557:87 , srv1080565524:88 , srv1096981633:88 , srv1183694488:88 , srv1250434833:88 , srv1575470588:88 , srv1781525340:88 , srv1796519974:88 , srv1936206570:88 , srv1994490565:88 , srv249354630:88 , srv613247825:88 , srv613745491:88 , srv1162557737:89 , srv1244194825:89 , srv1429114152:89 , srv1673579370:89 , srv182168835:89 , srv225980568:89 , srv482432863:89 , srv673123909:89 , srv69705026:89 , srv747986406:89 , srv1253940433:90 , srv906466163:90 , srv1416447314:91 , srv1635759831:92 , srv306552623:92 , srv486828065:92 , srv1512894696:93 , srv1169703964:98 , srv299007220:450 }
2023-02-12 03:59:21,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-12 03:59:21,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1575470588=36, srv1521223098=31, srv1737141428=41, srv1674725761=40, srv948625601=98, srv349599513=65, srv1320822633=20, srv1205537780=13, srv182168835=44, srv155685183=34, srv673123909=84, srv989124097=99, srv1512894696=30, srv457465505=71, srv107845026=2, srv1169703964=7, srv1553602073=33, srv476372959=73, srv185365900=45, srv1796519974=43, srv249354630=58, srv454090579=70, srv1994490565=50, srv763342768=89, srv298823962=61, srv482432863=74, srv1329163961=21, srv771693962=90, srv2019187920=53, srv1157300237=5, srv1415136137=24, srv1888934153=47, srv874297738=93, srv1080565524=3, srv125352517=17, srv304311585=63, srv402531003=66, srv564996557=79, srv1038192386=1, srv299007220=62, srv450032989=69, srv1162557737=6, srv622180497=83, srv1170431799=9, srv674898333=85, srv916322977=95, srv1781525340=42, srv2110637285=56, srv1416447314=25, srv2104069446=55, srv306552623=64, srv1635759831=38, srv747986406=88, srv613745491=82, srv1498756220=29, srv69705026=86, srv798597569=91, srv1429114152=26, srv1432234805=27, srv1250434833=16, srv2016347424=52, srv49332837=76, srv906466163=94, srv1882814044=46, srv1936206570=48, srv225980568=57, srv1025477327=0, srv418118198=68, srv1170295766=8, srv588701304=80, srv798830557=92, srv613247825=81, srv1178147673=11, srv930658942=97, srv1317275019=19, srv1631347951=37, srv404031598=67, srv921448887=96, srv2073699485=54, srv1565431045=35, srv1183694488=12, srv136131977=22, srv1996300=51, srv1396146779=23, srv498751518=77, srv272847005=60, srv1538584483=32, srv486828065=75, srv1244194825=14, srv460533288=72, srv1253940433=18, srv1480549808=28, srv1937703779=49, srv1250185792=15, srv743622037=87, srv552601984=78, srv1170545250=10, srv1673579370=39, srv1096981633=4, srv265771931=59} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-12 03:59:21,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-12 03:59:21,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-12 03:59:21,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-12 03:59:21,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-12 03:59:21,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-12 03:59:21,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-12 03:59:21,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-12 03:59:21,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-12 03:59:21,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-12 03:59:21,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-12 03:59:21,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-12 03:59:21,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-12 03:59:21,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-12 03:59:21,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-12 03:59:21,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-12 03:59:21,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-12 03:59:21,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-12 03:59:21,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-12 03:59:21,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-12 03:59:21,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-12 03:59:21,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-12 03:59:21,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-12 03:59:21,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-12 03:59:21,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-12 03:59:21,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-12 03:59:21,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-12 03:59:21,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-12 03:59:21,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-12 03:59:21,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-12 03:59:21,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-12 03:59:21,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-12 03:59:21,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-12 03:59:21,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-12 03:59:21,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-12 03:59:21,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-12 03:59:21,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-12 03:59:21,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-12 03:59:21,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-12 03:59:21,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-12 03:59:21,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-12 03:59:21,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-12 03:59:21,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-12 03:59:21,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-12 03:59:21,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-12 03:59:21,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-12 03:59:21,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-12 03:59:21,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-12 03:59:21,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-12 03:59:21,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-12 03:59:21,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-12 03:59:21,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-12 03:59:21,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-12 03:59:21,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-12 03:59:21,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-12 03:59:21,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-12 03:59:21,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-12 03:59:21,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-12 03:59:21,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-12 03:59:21,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-12 03:59:21,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-12 03:59:21,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-12 03:59:21,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-12 03:59:21,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-12 03:59:21,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-12 03:59:21,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-12 03:59:21,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-12 03:59:21,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-12 03:59:21,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-12 03:59:21,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-12 03:59:21,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-12 03:59:21,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-12 03:59:21,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-12 03:59:21,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-12 03:59:21,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-12 03:59:21,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-12 03:59:21,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-12 03:59:21,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-12 03:59:21,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-12 03:59:21,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-12 03:59:21,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-12 03:59:21,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-12 03:59:21,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-12 03:59:21,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-12 03:59:21,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-12 03:59:21,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-12 03:59:21,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-12 03:59:21,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-12 03:59:21,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-12 03:59:21,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-12 03:59:21,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-12 03:59:21,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-12 03:59:21,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-12 03:59:21,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-12 03:59:21,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-12 03:59:21,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-12 03:59:21,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-12 03:59:21,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-12 03:59:21,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-12 03:59:21,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-12 03:59:21,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-12 03:59:21,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-12 03:59:21,665 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040511753169985724, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0561543487776701, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040511753169985724, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.167, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23466666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-12 03:59:21,665 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-12 03:59:21,666 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17192053826626075, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040511753169985724, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0561543487776701, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040511753169985724, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.167, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23466666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-12 03:59:21,939 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 364 ms to try 5000 different iterations.  Found a solution that moves 2567 regions; Going from a computed imbalance of 0.17192053826626075 to a new imbalance of 0.014527538104295282. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023397645324029208); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03998877508045809, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2852222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023397645324029208); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011416666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04375, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-12 03:59:21,950 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1169703964:75 , srv1512894696:79 , srv1416447314:80 , srv1429114152:80 , srv306552623:80 , srv1080565524:81 , srv486828065:81 , srv673123909:81 , srv1183694488:82 , srv182168835:82 , srv1936206570:82 , srv747986406:82 , srv1178147673:83 , srv1250185792:83 , srv1635759831:83 , srv1673579370:83 , srv225980568:83 , srv249354630:83 , srv482432863:83 , srv613745491:83 , srv69705026:83 , srv798830557:83 , srv1244194825:84 , srv1253940433:84 , srv136131977:84 , srv1396146779:84 , srv1575470588:84 , srv1781525340:84 , srv1796519974:84 , srv1994490565:84 , srv2016347424:84 , srv906466163:84 , srv1250434833:85 , srv185365900:85 , srv1882814044:85 , srv460533288:85 , srv476372959:85 , srv674898333:85 , srv989124097:85 , srv1025477327:86 , srv1096981633:86 , srv1162557737:86 , srv1170295766:86 , srv1170431799:86 , srv1170545250:86 , srv1205537780:86 , srv1631347951:86 , srv1674725761:86 , srv1737141428:86 , srv1996300:86 , srv2019187920:86 , srv304311585:86 , srv349599513:86 , srv402531003:86 , srv454090579:86 , srv564996557:86 , srv613247825:86 , srv622180497:86 , srv771693962:86 , srv874297738:86 , srv1038192386:87 , srv1415136137:87 , srv1432234805:87 , srv1480549808:87 , srv1498756220:87 , srv1521223098:87 , srv1553602073:87 , srv155685183:87 , srv1937703779:87 , srv265771931:87 , srv272847005:87 , srv298823962:87 , srv49332837:87 , srv588701304:87 , srv763342768:87 , srv798597569:87 , srv921448887:87 , srv125352517:88 , srv1329163961:88 , srv1538584483:88 , srv1565431045:88 , srv2104069446:88 , srv404031598:88 , srv418118198:88 , srv450032989:88 , srv498751518:88 , srv930658942:88 , srv948625601:88 , srv1157300237:89 , srv1888934153:89 , srv2073699485:89 , srv457465505:89 , srv552601984:89 , srv743622037:89 , srv107845026:90 , srv1317275019:90 , srv1320822633:90 , srv916322977:90 , srv2110637285:91 , srv299007220:532 }
2023-02-12 03:59:21,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-12 03:59:21,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1575470588=36, srv1521223098=31, srv1737141428=41, srv1674725761=40, srv948625601=98, srv349599513=65, srv1320822633=20, srv1205537780=13, srv182168835=44, srv155685183=34, srv673123909=84, srv989124097=99, srv1512894696=30, srv457465505=71, srv107845026=2, srv1169703964=7, srv1553602073=33, srv476372959=73, srv185365900=45, srv1796519974=43, srv249354630=58, srv454090579=70, srv1994490565=50, srv763342768=89, srv298823962=61, srv482432863=74, srv1329163961=21, srv771693962=90, srv2019187920=53, srv1157300237=5, srv1415136137=24, srv1888934153=47, srv874297738=93, srv1080565524=3, srv125352517=17, srv304311585=63, srv402531003=66, srv564996557=79, srv1038192386=1, srv299007220=62, srv450032989=69, srv1162557737=6, srv622180497=83, srv1170431799=9, srv674898333=85, srv916322977=95, srv1781525340=42, srv2110637285=56, srv1416447314=25, srv2104069446=55, srv306552623=64, srv1635759831=38, srv747986406=88, srv613745491=82, srv1498756220=29, srv69705026=86, srv798597569=91, srv1429114152=26, srv1432234805=27, srv1250434833=16, srv2016347424=52, srv49332837=76, srv906466163=94, srv1882814044=46, srv1936206570=48, srv225980568=57, srv1025477327=0, srv418118198=68, srv1170295766=8, srv588701304=80, srv798830557=92, srv613247825=81, srv1178147673=11, srv930658942=97, srv1317275019=19, srv1631347951=37, srv404031598=67, srv921448887=96, srv2073699485=54, srv1565431045=35, srv1183694488=12, srv136131977=22, srv1996300=51, srv1396146779=23, srv498751518=77, srv272847005=60, srv1538584483=32, srv486828065=75, srv1244194825=14, srv460533288=72, srv1253940433=18, srv1480549808=28, srv1937703779=49, srv1250185792=15, srv743622037=87, srv552601984=78, srv1170545250=10, srv1673579370=39, srv1096981633=4, srv265771931=59} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-12 03:59:21,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-12 03:59:21,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-12 03:59:21,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-12 03:59:21,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-12 03:59:21,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-12 03:59:21,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-12 03:59:21,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-12 03:59:21,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-12 03:59:21,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-12 03:59:21,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-12 03:59:21,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-12 03:59:21,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-12 03:59:21,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-12 03:59:21,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-12 03:59:21,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-12 03:59:21,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-12 03:59:21,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-12 03:59:21,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-12 03:59:21,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-12 03:59:21,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-12 03:59:21,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-12 03:59:21,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-12 03:59:21,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-12 03:59:21,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-12 03:59:21,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-12 03:59:21,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-12 03:59:21,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-12 03:59:21,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-12 03:59:21,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-12 03:59:21,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-12 03:59:21,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-12 03:59:21,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-12 03:59:21,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-12 03:59:21,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-12 03:59:21,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-12 03:59:21,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-12 03:59:21,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-12 03:59:21,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-12 03:59:21,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-12 03:59:21,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-12 03:59:21,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-12 03:59:21,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-12 03:59:21,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-12 03:59:21,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-12 03:59:21,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-12 03:59:21,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-12 03:59:21,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-12 03:59:21,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-12 03:59:21,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-12 03:59:21,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-12 03:59:21,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-12 03:59:21,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-12 03:59:21,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-12 03:59:21,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-12 03:59:21,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-12 03:59:21,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-12 03:59:21,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-12 03:59:21,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-12 03:59:21,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-12 03:59:21,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-12 03:59:21,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-12 03:59:21,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-12 03:59:21,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-12 03:59:21,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-12 03:59:21,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-12 03:59:21,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-12 03:59:21,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-12 03:59:21,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-12 03:59:21,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-12 03:59:21,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-12 03:59:21,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-12 03:59:21,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-12 03:59:21,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-12 03:59:21,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-12 03:59:21,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-12 03:59:21,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-12 03:59:21,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-12 03:59:21,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-12 03:59:21,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-12 03:59:21,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-12 03:59:21,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-12 03:59:21,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-12 03:59:21,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-12 03:59:21,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-12 03:59:21,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-12 03:59:21,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-12 03:59:21,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-12 03:59:21,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-12 03:59:21,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-12 03:59:21,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-12 03:59:21,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-12 03:59:21,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-12 03:59:21,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-12 03:59:21,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-12 03:59:21,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-12 03:59:21,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-12 03:59:21,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-12 03:59:21,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-12 03:59:21,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-12 03:59:21,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-12 03:59:21,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-12 03:59:22,135 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023397645324029208); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03998877508045809, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023397645324029208); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011416666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04375, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-12 03:59:22,136 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-12 03:59:22,136 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.0145095635861182, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023397645324029208); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03998877508045809, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023397645324029208); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011416666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04375, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-12 03:59:22,336 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 320 ms to try 5000 different iterations.  Found a solution that moves 938 regions; Going from a computed imbalance of 0.0145095635861182 to a new imbalance of 9.57323464921061E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007547496857671467); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.013138304277253817); MoveCostFunction : (multiplier=7.0, imbalance=0.10422222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007547496857671467); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.005333333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-12 03:59:22,337 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1162557737:82 , srv1635759831:82 , srv2104069446:82 , srv552601984:82 , srv1096981633:83 , srv125352517:83 , srv1553602073:83 , srv265771931:83 , srv498751518:83 , srv107845026:84 , srv1169703964:84 , srv1244194825:84 , srv1253940433:84 , srv1320822633:84 , srv1432234805:84 , srv1888934153:84 , srv1996300:84 , srv454090579:84 , srv486828065:84 , srv613247825:84 , srv743622037:84 , srv798597569:84 , srv906466163:84 , srv916322977:84 , srv930658942:84 , srv1038192386:85 , srv1157300237:85 , srv1170295766:85 , srv1170545250:85 , srv1205537780:85 , srv1250434833:85 , srv1317275019:85 , srv1521223098:85 , srv155685183:85 , srv1565431045:85 , srv1575470588:85 , srv1674725761:85 , srv1781525340:85 , srv1796519974:85 , srv1937703779:85 , srv1994490565:85 , srv225980568:85 , srv272847005:85 , srv306552623:85 , srv404031598:85 , srv450032989:85 , srv457465505:85 , srv49332837:85 , srv564996557:85 , srv69705026:85 , srv763342768:85 , srv1025477327:86 , srv1170431799:86 , srv1329163961:86 , srv1415136137:86 , srv1416447314:86 , srv1498756220:86 , srv1512894696:86 , srv1631347951:86 , srv1673579370:86 , srv1737141428:86 , srv2019187920:86 , srv2073699485:86 , srv249354630:86 , srv402531003:86 , srv460533288:86 , srv476372959:86 , srv482432863:86 , srv613745491:86 , srv622180497:86 , srv674898333:86 , srv747986406:86 , srv874297738:86 , srv921448887:86 , srv948625601:86 , srv1250185792:87 , srv136131977:87 , srv1396146779:87 , srv1480549808:87 , srv1538584483:87 , srv182168835:87 , srv185365900:87 , srv1882814044:87 , srv1936206570:87 , srv2016347424:87 , srv2110637285:87 , srv298823962:87 , srv304311585:87 , srv349599513:87 , srv588701304:87 , srv673123909:87 , srv798830557:87 , srv1080565524:88 , srv1178147673:88 , srv1183694488:88 , srv418118198:88 , srv771693962:88 , srv989124097:88 , srv1429114152:89 , srv299007220:543 }
2023-02-12 03:59:22,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-12 03:59:22,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1575470588=36, srv1521223098=31, srv1737141428=41, srv1674725761=40, srv948625601=98, srv349599513=65, srv1320822633=20, srv1205537780=13, srv182168835=44, srv155685183=34, srv673123909=84, srv989124097=99, srv1512894696=30, srv457465505=71, srv107845026=2, srv1169703964=7, srv1553602073=33, srv476372959=73, srv185365900=45, srv1796519974=43, srv249354630=58, srv454090579=70, srv1994490565=50, srv763342768=89, srv298823962=61, srv482432863=74, srv1329163961=21, srv771693962=90, srv2019187920=53, srv1157300237=5, srv1415136137=24, srv1888934153=47, srv874297738=93, srv1080565524=3, srv125352517=17, srv304311585=63, srv402531003=66, srv564996557=79, srv1038192386=1, srv299007220=62, srv450032989=69, srv1162557737=6, srv622180497=83, srv1170431799=9, srv674898333=85, srv916322977=95, srv1781525340=42, srv2110637285=56, srv1416447314=25, srv2104069446=55, srv306552623=64, srv1635759831=38, srv747986406=88, srv613745491=82, srv1498756220=29, srv69705026=86, srv798597569=91, srv1429114152=26, srv1432234805=27, srv1250434833=16, srv2016347424=52, srv49332837=76, srv906466163=94, srv1882814044=46, srv1936206570=48, srv225980568=57, srv1025477327=0, srv418118198=68, srv1170295766=8, srv588701304=80, srv798830557=92, srv613247825=81, srv1178147673=11, srv930658942=97, srv1317275019=19, srv1631347951=37, srv404031598=67, srv921448887=96, srv2073699485=54, srv1565431045=35, srv1183694488=12, srv136131977=22, srv1996300=51, srv1396146779=23, srv498751518=77, srv272847005=60, srv1538584483=32, srv486828065=75, srv1244194825=14, srv460533288=72, srv1253940433=18, srv1480549808=28, srv1937703779=49, srv1250185792=15, srv743622037=87, srv552601984=78, srv1170545250=10, srv1673579370=39, srv1096981633=4, srv265771931=59} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-12 03:59:22,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-12 03:59:22,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-12 03:59:22,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-12 03:59:22,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-12 03:59:22,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-12 03:59:22,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-12 03:59:22,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-12 03:59:22,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-12 03:59:22,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-12 03:59:22,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-12 03:59:22,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-12 03:59:22,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-12 03:59:22,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-12 03:59:22,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-12 03:59:22,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-12 03:59:22,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-12 03:59:22,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-12 03:59:22,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-12 03:59:22,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-12 03:59:22,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-12 03:59:22,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-12 03:59:22,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-12 03:59:22,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-12 03:59:22,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-12 03:59:22,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-12 03:59:22,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-12 03:59:22,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-12 03:59:22,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-12 03:59:22,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-12 03:59:22,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-12 03:59:22,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-12 03:59:22,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-12 03:59:22,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-12 03:59:22,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-12 03:59:22,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-12 03:59:22,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-12 03:59:22,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-12 03:59:22,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-12 03:59:22,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-12 03:59:22,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-12 03:59:22,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-12 03:59:22,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-12 03:59:22,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-12 03:59:22,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-12 03:59:22,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-12 03:59:22,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-12 03:59:22,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-12 03:59:22,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-12 03:59:22,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-12 03:59:22,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-12 03:59:22,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-12 03:59:22,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-12 03:59:22,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-12 03:59:22,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-12 03:59:22,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-12 03:59:22,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-12 03:59:22,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-12 03:59:22,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-12 03:59:22,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-12 03:59:22,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-12 03:59:22,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-12 03:59:22,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-12 03:59:22,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-12 03:59:22,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-12 03:59:22,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-12 03:59:22,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-12 03:59:22,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-12 03:59:22,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-12 03:59:22,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-12 03:59:22,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-12 03:59:22,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-12 03:59:22,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-12 03:59:22,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-12 03:59:22,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-12 03:59:22,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-12 03:59:22,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-12 03:59:22,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-12 03:59:22,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-12 03:59:22,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-12 03:59:22,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-12 03:59:22,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-12 03:59:22,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-12 03:59:22,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-12 03:59:22,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-12 03:59:22,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-12 03:59:22,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-12 03:59:22,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-12 03:59:22,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-12 03:59:22,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-12 03:59:22,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-12 03:59:22,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-12 03:59:22,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-12 03:59:22,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-12 03:59:22,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-12 03:59:22,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-12 03:59:22,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-12 03:59:22,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-12 03:59:22,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-12 03:59:22,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-12 03:59:22,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-12 03:59:22,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-12 03:59:22,441 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007547496857671467); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.013138304277253817); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007547496857671467); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.005333333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-12 03:59:22,441 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-12 03:59:22,441 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=9.507554485400321E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007547496857671467); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.013138304277253817); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007547496857671467); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.005333333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-12 03:59:22,672 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 307 ms to try 5000 different iterations.  Found a solution that moves 312 regions; Going from a computed imbalance of 9.507554485400321E-4 to a new imbalance of 2.10029528226639E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.23782800878924E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0019534423844256277); MoveCostFunction : (multiplier=7.0, imbalance=0.034666666666666665, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.23782800878924E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-12 03:59:22,673 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1253940433:83 , srv136131977:83 , srv155685183:83 , srv2073699485:83 , srv771693962:83 , srv107845026:84 , srv1170295766:84 , srv1170431799:84 , srv1183694488:84 , srv1429114152:84 , srv1480549808:84 , srv1512894696:84 , srv1521223098:84 , srv1538584483:84 , srv1565431045:84 , srv1631347951:84 , srv1673579370:84 , srv1737141428:84 , srv182168835:84 , srv1882814044:84 , srv2016347424:84 , srv298823962:84 , srv304311585:84 , srv418118198:84 , srv454090579:84 , srv482432863:84 , srv49332837:84 , srv564996557:84 , srv588701304:84 , srv613247825:84 , srv622180497:84 , srv743622037:84 , srv874297738:84 , srv906466163:84 , srv921448887:84 , srv948625601:84 , srv1025477327:85 , srv1038192386:85 , srv1080565524:85 , srv1096981633:85 , srv1157300237:85 , srv1169703964:85 , srv1178147673:85 , srv1205537780:85 , srv1244194825:85 , srv1250185792:85 , srv1250434833:85 , srv125352517:85 , srv1317275019:85 , srv1320822633:85 , srv1329163961:85 , srv1396146779:85 , srv1416447314:85 , srv1432234805:85 , srv1498756220:85 , srv1553602073:85 , srv1635759831:85 , srv1674725761:85 , srv1781525340:85 , srv1796519974:85 , srv185365900:85 , srv1888934153:85 , srv1936206570:85 , srv1937703779:85 , srv1994490565:85 , srv1996300:85 , srv2104069446:85 , srv2110637285:85 , srv225980568:85 , srv249354630:85 , srv265771931:85 , srv272847005:85 , srv306552623:85 , srv349599513:85 , srv402531003:85 , srv404031598:85 , srv450032989:85 , srv457465505:85 , srv460533288:85 , srv476372959:85 , srv486828065:85 , srv498751518:85 , srv552601984:85 , srv613745491:85 , srv673123909:85 , srv69705026:85 , srv798597569:85 , srv798830557:85 , srv916322977:85 , srv930658942:85 , srv989124097:85 , srv1162557737:86 , srv1170545250:86 , srv1575470588:86 , srv2019187920:86 , srv674898333:86 , srv747986406:86 , srv763342768:86 , srv1415136137:87 , srv299007220:617 }
2023-02-12 03:59:22,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-12 03:59:22,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1575470588=36, srv1521223098=31, srv1737141428=41, srv1674725761=40, srv948625601=98, srv349599513=65, srv1320822633=20, srv1205537780=13, srv182168835=44, srv155685183=34, srv673123909=84, srv989124097=99, srv1512894696=30, srv457465505=71, srv107845026=2, srv1169703964=7, srv1553602073=33, srv476372959=73, srv185365900=45, srv1796519974=43, srv249354630=58, srv454090579=70, srv1994490565=50, srv763342768=89, srv298823962=61, srv482432863=74, srv1329163961=21, srv771693962=90, srv2019187920=53, srv1157300237=5, srv1415136137=24, srv1888934153=47, srv874297738=93, srv1080565524=3, srv125352517=17, srv304311585=63, srv402531003=66, srv564996557=79, srv1038192386=1, srv299007220=62, srv450032989=69, srv1162557737=6, srv622180497=83, srv1170431799=9, srv674898333=85, srv916322977=95, srv1781525340=42, srv2110637285=56, srv1416447314=25, srv2104069446=55, srv306552623=64, srv1635759831=38, srv747986406=88, srv613745491=82, srv1498756220=29, srv69705026=86, srv798597569=91, srv1429114152=26, srv1432234805=27, srv1250434833=16, srv2016347424=52, srv49332837=76, srv906466163=94, srv1882814044=46, srv1936206570=48, srv225980568=57, srv1025477327=0, srv418118198=68, srv1170295766=8, srv588701304=80, srv798830557=92, srv613247825=81, srv1178147673=11, srv930658942=97, srv1317275019=19, srv1631347951=37, srv404031598=67, srv921448887=96, srv2073699485=54, srv1565431045=35, srv1183694488=12, srv136131977=22, srv1996300=51, srv1396146779=23, srv498751518=77, srv272847005=60, srv1538584483=32, srv486828065=75, srv1244194825=14, srv460533288=72, srv1253940433=18, srv1480549808=28, srv1937703779=49, srv1250185792=15, srv743622037=87, srv552601984=78, srv1170545250=10, srv1673579370=39, srv1096981633=4, srv265771931=59} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-12 03:59:22,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-12 03:59:22,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-12 03:59:22,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-12 03:59:22,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-12 03:59:22,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-12 03:59:22,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-12 03:59:22,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-12 03:59:22,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-12 03:59:22,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-12 03:59:22,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-12 03:59:22,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-12 03:59:22,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-12 03:59:22,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-12 03:59:22,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-12 03:59:22,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-12 03:59:22,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-12 03:59:22,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-12 03:59:22,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-12 03:59:22,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-12 03:59:22,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-12 03:59:22,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-12 03:59:22,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-12 03:59:22,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-12 03:59:22,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-12 03:59:22,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-12 03:59:22,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-12 03:59:22,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-12 03:59:22,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-12 03:59:22,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-12 03:59:22,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-12 03:59:22,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-12 03:59:22,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-12 03:59:22,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-12 03:59:22,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-12 03:59:22,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-12 03:59:22,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-12 03:59:22,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-12 03:59:22,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-12 03:59:22,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-12 03:59:22,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-12 03:59:22,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-12 03:59:22,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-12 03:59:22,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-12 03:59:22,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-12 03:59:22,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-12 03:59:22,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-12 03:59:22,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-12 03:59:22,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-12 03:59:22,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-12 03:59:22,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-12 03:59:22,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-12 03:59:22,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-12 03:59:22,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-12 03:59:22,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-12 03:59:22,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-12 03:59:22,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-12 03:59:22,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-12 03:59:22,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-12 03:59:22,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-12 03:59:22,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-12 03:59:22,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-12 03:59:22,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-12 03:59:22,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-12 03:59:22,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-12 03:59:22,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-12 03:59:22,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-12 03:59:22,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-12 03:59:22,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-12 03:59:22,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-12 03:59:22,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-12 03:59:22,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-12 03:59:22,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-12 03:59:22,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-12 03:59:22,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-12 03:59:22,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-12 03:59:22,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-12 03:59:22,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-12 03:59:22,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-12 03:59:22,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-12 03:59:22,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-12 03:59:22,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-12 03:59:22,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-12 03:59:22,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-12 03:59:22,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-12 03:59:22,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-12 03:59:22,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-12 03:59:22,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-12 03:59:22,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-12 03:59:22,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-12 03:59:22,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-12 03:59:22,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-12 03:59:22,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-12 03:59:22,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-12 03:59:22,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-12 03:59:22,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-12 03:59:22,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-12 03:59:22,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-12 03:59:22,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-12 03:59:22,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-12 03:59:22,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-12 03:59:22,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-12 03:59:22,807 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.8818282128760873E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.23782800878924E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0019534423844256277); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.23782800878924E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-12 03:59:22,808 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1253940433:83 , srv136131977:83 , srv155685183:83 , srv2073699485:83 , srv771693962:83 , srv107845026:84 , srv1170295766:84 , srv1170431799:84 , srv1183694488:84 , srv1429114152:84 , srv1480549808:84 , srv1512894696:84 , srv1521223098:84 , srv1538584483:84 , srv1565431045:84 , srv1631347951:84 , srv1673579370:84 , srv1737141428:84 , srv182168835:84 , srv1882814044:84 , srv2016347424:84 , srv298823962:84 , srv304311585:84 , srv418118198:84 , srv454090579:84 , srv482432863:84 , srv49332837:84 , srv564996557:84 , srv588701304:84 , srv613247825:84 , srv622180497:84 , srv743622037:84 , srv874297738:84 , srv906466163:84 , srv921448887:84 , srv948625601:84 , srv1025477327:85 , srv1038192386:85 , srv1080565524:85 , srv1096981633:85 , srv1157300237:85 , srv1169703964:85 , srv1178147673:85 , srv1205537780:85 , srv1244194825:85 , srv1250185792:85 , srv1250434833:85 , srv125352517:85 , srv1317275019:85 , srv1320822633:85 , srv1329163961:85 , srv1396146779:85 , srv1416447314:85 , srv1432234805:85 , srv1498756220:85 , srv1553602073:85 , srv1635759831:85 , srv1674725761:85 , srv1781525340:85 , srv1796519974:85 , srv185365900:85 , srv1888934153:85 , srv1936206570:85 , srv1937703779:85 , srv1994490565:85 , srv1996300:85 , srv2104069446:85 , srv2110637285:85 , srv225980568:85 , srv249354630:85 , srv265771931:85 , srv272847005:85 , srv306552623:85 , srv349599513:85 , srv402531003:85 , srv404031598:85 , srv450032989:85 , srv457465505:85 , srv460533288:85 , srv476372959:85 , srv486828065:85 , srv498751518:85 , srv552601984:85 , srv613745491:85 , srv673123909:85 , srv69705026:85 , srv798597569:85 , srv798830557:85 , srv916322977:85 , srv930658942:85 , srv989124097:85 , srv1162557737:86 , srv1170545250:86 , srv1575470588:86 , srv2019187920:86 , srv674898333:86 , srv747986406:86 , srv763342768:86 , srv1415136137:87 , srv299007220:617 }
2023-02-12 03:59:22,836 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=271 (was 271), ProcessCount=164 (was 164), AvailableMemoryMB=4073 (was 4574)
2023-02-12 03:59:22,843 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=271, ProcessCount=164, AvailableMemoryMB=4072
2023-02-12 03:59:22,844 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-12 03:59:22,844 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-02-12 03:59:22,851 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1019709983:84 , srv1096530326:84 , srv11238587:84 , srv1424168448:84 , srv1433308564:84 , srv1465277620:84 , srv1528783192:84 , srv1692851186:84 , srv1909960389:84 , srv1912092684:84 , srv2033641582:84 , srv2104003331:84 , srv2136050642:84 , srv260675132:84 , srv293071609:84 , srv334533996:84 , srv370178206:84 , srv392305924:84 , srv409637568:84 , srv409676617:84 , srv466588763:84 , srv649154674:84 , srv68271836:84 , srv769074795:84 , srv77878954:84 , srv782272626:84 , srv818757398:84 , srv862245254:84 , srv984774188:84 , srv2025120773:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-02-12 03:59:22,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-12 03:59:22,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv68271836=23, srv984774188=29, srv1528783192=6, srv1433308564=4, srv334533996=16, srv1424168448=3, srv1019709983=0, srv2136050642=13, srv2104003331=12, srv293071609=15, srv1465277620=5, srv2033641582=11, srv2025120773=10, srv1096530326=1, srv1692851186=7, srv260675132=14, srv1912092684=9, srv649154674=22, srv466588763=21, srv409676617=20, srv409637568=19, srv818757398=27, srv862245254=28, srv370178206=17, srv769074795=24, srv1909960389=8, srv77878954=25, srv11238587=2, srv392305924=18, srv782272626=26} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-12 03:59:22,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-12 03:59:22,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-12 03:59:22,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-12 03:59:22,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-12 03:59:22,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-12 03:59:22,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-12 03:59:22,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-12 03:59:22,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-12 03:59:22,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-12 03:59:22,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-12 03:59:22,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-12 03:59:22,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-12 03:59:22,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-12 03:59:22,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-12 03:59:22,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-12 03:59:22,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-12 03:59:22,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-12 03:59:22,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-12 03:59:22,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-12 03:59:22,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-12 03:59:22,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-12 03:59:22,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-12 03:59:22,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-12 03:59:22,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-12 03:59:22,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-12 03:59:22,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-12 03:59:22,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-12 03:59:22,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-12 03:59:22,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-12 03:59:22,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-12 03:59:22,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-02-12 03:59:22,875 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.106222656878876, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-12 03:59:22,875 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912527747387617, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.106222656878876, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-02-12 04:01:22,865 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 9361439 different iterations.  Found a solution that moves 1849 regions; Going from a computed imbalance of 0.9912527747387617 to a new imbalance of 4.3156582404131396E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6848148148148148, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-12 04:01:22,871 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1019709983:90 , srv1096530326:90 , srv11238587:90 , srv1424168448:90 , srv1433308564:90 , srv1465277620:90 , srv1528783192:90 , srv1692851186:90 , srv1909960389:90 , srv1912092684:90 , srv2025120773:90 , srv2033641582:90 , srv2104003331:90 , srv2136050642:90 , srv260675132:90 , srv293071609:90 , srv334533996:90 , srv370178206:90 , srv392305924:90 , srv409637568:90 , srv409676617:90 , srv466588763:90 , srv649154674:90 , srv68271836:90 , srv769074795:90 , srv77878954:90 , srv782272626:90 , srv818757398:90 , srv862245254:90 , srv984774188:90 }
2023-02-12 04:01:22,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-12 04:01:22,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv68271836=23, srv984774188=29, srv1528783192=6, srv1433308564=4, srv334533996=16, srv1424168448=3, srv1019709983=0, srv2136050642=13, srv2104003331=12, srv293071609=15, srv1465277620=5, srv2033641582=11, srv2025120773=10, srv1096530326=1, srv1692851186=7, srv260675132=14, srv1912092684=9, srv649154674=22, srv466588763=21, srv409676617=20, srv409637568=19, srv818757398=27, srv862245254=28, srv370178206=17, srv769074795=24, srv1909960389=8, srv77878954=25, srv11238587=2, srv392305924=18, srv782272626=26} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-12 04:01:22,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-12 04:01:22,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-12 04:01:22,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-12 04:01:22,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-12 04:01:22,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-12 04:01:22,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-12 04:01:22,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-12 04:01:22,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-12 04:01:22,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-12 04:01:22,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-12 04:01:22,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-12 04:01:22,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-12 04:01:22,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-12 04:01:22,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-12 04:01:22,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-12 04:01:22,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-12 04:01:22,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-12 04:01:22,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-12 04:01:22,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-12 04:01:22,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-12 04:01:22,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-12 04:01:22,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-12 04:01:22,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-12 04:01:22,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-12 04:01:22,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-12 04:01:22,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-12 04:01:22,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-12 04:01:22,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-12 04:01:22,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-12 04:01:22,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-12 04:01:22,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-02-12 04:01:22,887 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-12 04:01:22,888 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1019709983:90 , srv1096530326:90 , srv11238587:90 , srv1424168448:90 , srv1433308564:90 , srv1465277620:90 , srv1528783192:90 , srv1692851186:90 , srv1909960389:90 , srv1912092684:90 , srv2025120773:90 , srv2033641582:90 , srv2104003331:90 , srv2136050642:90 , srv260675132:90 , srv293071609:90 , srv334533996:90 , srv370178206:90 , srv392305924:90 , srv409637568:90 , srv409676617:90 , srv466588763:90 , srv649154674:90 , srv68271836:90 , srv769074795:90 , srv77878954:90 , srv782272626:90 , srv818757398:90 , srv862245254:90 , srv984774188:90 }
2023-02-12 04:01:22,899 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=189 (was 271), ProcessCount=165 (was 164) - ProcessCount LEAK? -, AvailableMemoryMB=4081 (was 4072) - AvailableMemoryMB LEAK? -