2023-02-19 12:53:59,807 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-02-19 12:54:00,096 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-02-19 12:54:00,197 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-02-19 12:54:00,262 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-19 12:54:00,270 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-02-19 12:54:00,283 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=325, ProcessCount=170, AvailableMemoryMB=10703
2023-02-19 12:54:00,285 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-19 12:54:00,286 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-02-19 12:54:00,407 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1009921656:84 , srv1016919167:84 , srv1017732837:84 , srv102353437:84 , srv1023756996:84 , srv103830899:84 , srv1046137911:84 , srv1059252006:84 , srv1075036867:84 , srv114799786:84 , srv1186878549:84 , srv1207701725:84 , srv1211273243:84 , srv1218995925:84 , srv1240406676:84 , srv1261719314:84 , srv1273173732:84 , srv1275683056:84 , srv1298241979:84 , srv1300339401:84 , srv1311866145:84 , srv132851632:84 , srv1366595432:84 , srv1366860215:84 , srv1376742936:84 , srv1390972464:84 , srv140895142:84 , srv1429261044:84 , srv1436922347:84 , srv1441522114:84 , srv1445241560:84 , srv1468433128:84 , srv1491198008:84 , srv1518418677:84 , srv1528452956:84 , srv1567225327:84 , srv1585204045:84 , srv1595086038:84 , srv1611297795:84 , srv1624254264:84 , srv1637040790:84 , srv1637089338:84 , srv1863002799:84 , srv188657073:84 , srv193237065:84 , srv1935331893:84 , srv1956549818:84 , srv1976628776:84 , srv2019977164:84 , srv2039048773:84 , srv2041030926:84 , srv2045289259:84 , srv2060876183:84 , srv2083323731:84 , srv2098740511:84 , srv2121622720:84 , srv212659528:84 , srv2129024947:84 , srv2138145618:84 , srv2140601606:84 , srv232690724:84 , srv233859433:84 , srv261363240:84 , srv27263691:84 , srv284243024:84 , srv374288556:84 , srv396896953:84 , srv41596212:84 , srv438221467:84 , srv446472230:84 , srv468994665:84 , srv530976730:84 , srv552030613:84 , srv562664628:84 , srv562862418:84 , srv573365634:84 , srv577871485:84 , srv61853811:84 , srv626568787:84 , srv633318387:84 , srv650335536:84 , srv678409492:84 , srv678997394:84 , srv682558210:84 , srv724186438:84 , srv725461257:84 , srv768340453:84 , srv788900359:84 , srv793603965:84 , srv9106584:84 , srv933015772:84 , srv936233759:84 , srv937506904:84 , srv966770169:84 , srv969513038:84 , srv979260650:84 , srv98143725:84 , srv992319801:84 , srv994996126:84 , srv362182774:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-02-19 12:54:00,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-19 12:54:00,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv562862418=75, srv1935331893=45, srv362182774=65, srv212659528=56, srv132851632=21, srv1518418677=33, srv232690724=60, srv2140601606=59, srv793603965=89, srv1390972464=25, srv261363240=62, srv1298241979=18, srv678409492=82, srv374288556=66, srv1075036867=8, srv1275683056=17, srv193237065=44, srv979260650=96, srv1023756996=4, srv933015772=91, srv61853811=78, srv552030613=73, srv1311866145=20, srv1637040790=40, srv98143725=97, srv1211273243=12, srv446472230=70, srv1366595432=22, srv396896953=67, srv1445241560=30, srv626568787=79, srv1567225327=35, srv41596212=68, srv1436922347=28, srv1585204045=36, srv1976628776=47, srv2098740511=54, srv788900359=88, srv682558210=84, srv102353437=3, srv2083323731=53, srv1595086038=37, srv1009921656=0, srv1468433128=31, srv1624254264=39, srv633318387=80, srv1261719314=15, srv936233759=92, srv1207701725=11, srv1300339401=19, srv1429261044=27, srv140895142=26, srv678997394=83, srv969513038=95, srv650335536=81, srv1366860215=23, srv994996126=99, srv2045289259=51, srv468994665=71, srv1218995925=13, srv2019977164=48, srv966770169=94, srv725461257=86, srv1637089338=41, srv284243024=64, srv724186438=85, srv992319801=98, srv1017732837=2, srv2129024947=57, srv937506904=93, srv233859433=61, srv1186878549=10, srv573365634=76, srv1956549818=46, srv530976730=72, srv103830899=5, srv1611297795=38, srv2121622720=55, srv27263691=63, srv1016919167=1, srv188657073=43, srv1441522114=29, srv562664628=74, srv2060876183=52, srv1273173732=16, srv2138145618=58, srv9106584=90, srv1240406676=14, srv1046137911=6, srv2041030926=50, srv1863002799=42, srv768340453=87, srv1376742936=24, srv438221467=69, srv577871485=77, srv114799786=9, srv2039048773=49, srv1059252006=7, srv1528452956=34, srv1491198008=32} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-19 12:54:00,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-19 12:54:00,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-19 12:54:00,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-19 12:54:00,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-19 12:54:00,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-19 12:54:00,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-19 12:54:00,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-19 12:54:00,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-19 12:54:00,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-19 12:54:00,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-19 12:54:00,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-19 12:54:00,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-19 12:54:00,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-19 12:54:00,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-19 12:54:00,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-19 12:54:00,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-19 12:54:00,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-19 12:54:00,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-19 12:54:00,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-19 12:54:00,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-19 12:54:00,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-19 12:54:00,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-19 12:54:00,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-19 12:54:00,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-19 12:54:00,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-19 12:54:00,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-19 12:54:00,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-19 12:54:00,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-19 12:54:00,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-19 12:54:00,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-19 12:54:00,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-19 12:54:00,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-19 12:54:00,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-19 12:54:00,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-19 12:54:00,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-19 12:54:00,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-19 12:54:00,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-19 12:54:00,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-19 12:54:00,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-19 12:54:00,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-19 12:54:00,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-19 12:54:00,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-19 12:54:00,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-19 12:54:00,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-19 12:54:00,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-19 12:54:00,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-19 12:54:00,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-19 12:54:00,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-19 12:54:00,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-19 12:54:00,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-19 12:54:00,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-19 12:54:00,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-19 12:54:00,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-19 12:54:00,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-19 12:54:00,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-19 12:54:00,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-19 12:54:00,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-19 12:54:00,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-19 12:54:00,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-19 12:54:00,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-19 12:54:00,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-19 12:54:00,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-19 12:54:00,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-19 12:54:00,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-19 12:54:00,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-19 12:54:00,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-19 12:54:00,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-19 12:54:00,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-19 12:54:00,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-19 12:54:00,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-19 12:54:00,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-19 12:54:00,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-19 12:54:00,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-19 12:54:00,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-19 12:54:00,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-19 12:54:00,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-19 12:54:00,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-19 12:54:00,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-19 12:54:00,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-19 12:54:00,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-19 12:54:00,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-19 12:54:00,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-19 12:54:00,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-19 12:54:00,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-19 12:54:00,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-19 12:54:00,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-19 12:54:00,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-19 12:54:00,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-19 12:54:00,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-19 12:54:00,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-19 12:54:00,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-19 12:54:00,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-19 12:54:00,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-19 12:54:00,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-19 12:54:00,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-19 12:54:00,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-19 12:54:00,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-19 12:54:00,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-19 12:54:00,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-19 12:54:00,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-19 12:54:00,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-19 12:54:00,870 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-19 12:54:00,870 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-19 12:54:00,870 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-19 12:54:01,248 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 696 ms to try 5000 different iterations.  Found a solution that moves 4703 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17579862653442602. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04165428173586767, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05581555322705893, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5225555555555556, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04165428173586767, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17125, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23483333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-19 12:54:01,272 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1211273243:80 , srv682558210:80 , srv1491198008:81 , srv573365634:81 , srv936233759:81 , srv1275683056:82 , srv1311866145:82 , srv1585204045:82 , srv446472230:82 , srv577871485:82 , srv966770169:82 , srv2019977164:83 , srv2121622720:83 , srv284243024:83 , srv102353437:84 , srv1300339401:84 , srv1366595432:84 , srv1429261044:84 , srv1436922347:84 , srv1611297795:84 , srv1935331893:84 , srv232690724:84 , srv261363240:84 , srv374288556:84 , srv530976730:84 , srv678997394:84 , srv979260650:84 , srv1017732837:85 , srv1059252006:85 , srv1376742936:85 , srv1445241560:85 , srv1528452956:85 , srv1637040790:85 , srv193237065:85 , srv2045289259:85 , srv212659528:85 , srv2138145618:85 , srv27263691:85 , srv41596212:85 , srv438221467:85 , srv562862418:85 , srv788900359:85 , srv98143725:85 , srv1009921656:86 , srv1075036867:86 , srv114799786:86 , srv1218995925:86 , srv140895142:86 , srv1441522114:86 , srv1595086038:86 , srv2098740511:86 , srv768340453:86 , srv937506904:86 , srv994996126:86 , srv1863002799:87 , srv188657073:87 , srv2041030926:87 , srv2060876183:87 , srv2083323731:87 , srv396896953:87 , srv562664628:87 , srv9106584:87 , srv933015772:87 , srv992319801:87 , srv1016919167:88 , srv1186878549:88 , srv1207701725:88 , srv1261719314:88 , srv1273173732:88 , srv1366860215:88 , srv1468433128:88 , srv1518418677:88 , srv1624254264:88 , srv1637089338:88 , srv1976628776:88 , srv2039048773:88 , srv468994665:88 , srv626568787:88 , srv650335536:88 , srv793603965:88 , srv1046137911:89 , srv1567225327:89 , srv1956549818:89 , srv233859433:89 , srv61853811:89 , srv724186438:89 , srv1023756996:90 , srv103830899:90 , srv2129024947:90 , srv2140601606:90 , srv633318387:90 , srv678409492:90 , srv969513038:90 , srv725461257:91 , srv1240406676:92 , srv1298241979:92 , srv132851632:92 , srv1390972464:93 , srv552030613:96 , srv362182774:460 }
2023-02-19 12:54:01,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-19 12:54:01,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv562862418=75, srv1935331893=45, srv362182774=65, srv212659528=56, srv132851632=21, srv1518418677=33, srv232690724=60, srv2140601606=59, srv793603965=89, srv1390972464=25, srv261363240=62, srv1298241979=18, srv678409492=82, srv374288556=66, srv1075036867=8, srv1275683056=17, srv193237065=44, srv979260650=96, srv1023756996=4, srv933015772=91, srv61853811=78, srv552030613=73, srv1311866145=20, srv1637040790=40, srv98143725=97, srv1211273243=12, srv446472230=70, srv1366595432=22, srv396896953=67, srv1445241560=30, srv626568787=79, srv1567225327=35, srv41596212=68, srv1436922347=28, srv1585204045=36, srv1976628776=47, srv2098740511=54, srv788900359=88, srv682558210=84, srv102353437=3, srv2083323731=53, srv1595086038=37, srv1009921656=0, srv1468433128=31, srv1624254264=39, srv633318387=80, srv1261719314=15, srv936233759=92, srv1207701725=11, srv1300339401=19, srv1429261044=27, srv140895142=26, srv678997394=83, srv969513038=95, srv650335536=81, srv1366860215=23, srv994996126=99, srv2045289259=51, srv468994665=71, srv1218995925=13, srv2019977164=48, srv966770169=94, srv725461257=86, srv1637089338=41, srv284243024=64, srv724186438=85, srv992319801=98, srv1017732837=2, srv2129024947=57, srv937506904=93, srv233859433=61, srv1186878549=10, srv573365634=76, srv1956549818=46, srv530976730=72, srv103830899=5, srv1611297795=38, srv2121622720=55, srv27263691=63, srv1016919167=1, srv188657073=43, srv1441522114=29, srv562664628=74, srv2060876183=52, srv1273173732=16, srv2138145618=58, srv9106584=90, srv1240406676=14, srv1046137911=6, srv2041030926=50, srv1863002799=42, srv768340453=87, srv1376742936=24, srv438221467=69, srv577871485=77, srv114799786=9, srv2039048773=49, srv1059252006=7, srv1528452956=34, srv1491198008=32} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-19 12:54:01,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-19 12:54:01,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-19 12:54:01,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-19 12:54:01,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-19 12:54:01,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-19 12:54:01,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-19 12:54:01,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-19 12:54:01,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-19 12:54:01,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-19 12:54:01,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-19 12:54:01,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-19 12:54:01,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-19 12:54:01,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-19 12:54:01,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-19 12:54:01,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-19 12:54:01,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-19 12:54:01,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-19 12:54:01,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-19 12:54:01,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-19 12:54:01,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-19 12:54:01,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-19 12:54:01,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-19 12:54:01,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-19 12:54:01,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-19 12:54:01,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-19 12:54:01,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-19 12:54:01,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-19 12:54:01,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-19 12:54:01,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-19 12:54:01,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-19 12:54:01,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-19 12:54:01,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-19 12:54:01,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-19 12:54:01,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-19 12:54:01,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-19 12:54:01,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-19 12:54:01,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-19 12:54:01,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-19 12:54:01,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-19 12:54:01,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-19 12:54:01,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-19 12:54:01,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-19 12:54:01,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-19 12:54:01,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-19 12:54:01,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-19 12:54:01,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-19 12:54:01,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-19 12:54:01,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-19 12:54:01,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-19 12:54:01,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-19 12:54:01,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-19 12:54:01,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-19 12:54:01,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-19 12:54:01,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-19 12:54:01,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-19 12:54:01,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-19 12:54:01,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-19 12:54:01,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-19 12:54:01,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-19 12:54:01,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-19 12:54:01,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-19 12:54:01,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-19 12:54:01,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-19 12:54:01,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-19 12:54:01,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-19 12:54:01,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-19 12:54:01,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-19 12:54:01,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-19 12:54:01,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-19 12:54:01,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-19 12:54:01,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-19 12:54:01,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-19 12:54:01,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-19 12:54:01,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-19 12:54:01,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-19 12:54:01,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-19 12:54:01,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-19 12:54:01,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-19 12:54:01,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-19 12:54:01,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-19 12:54:01,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-19 12:54:01,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-19 12:54:01,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-19 12:54:01,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-19 12:54:01,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-19 12:54:01,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-19 12:54:01,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-19 12:54:01,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-19 12:54:01,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-19 12:54:01,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-19 12:54:01,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-19 12:54:01,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-19 12:54:01,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-19 12:54:01,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-19 12:54:01,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-19 12:54:01,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-19 12:54:01,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-19 12:54:01,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-19 12:54:01,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-19 12:54:01,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-19 12:54:01,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-19 12:54:01,462 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04165428173586767, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05581555322705893, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04165428173586767, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17125, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23483333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-19 12:54:01,463 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-19 12:54:01,463 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17576569542457532, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04165428173586767, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05581555322705893, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04165428173586767, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17125, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23483333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-19 12:54:01,821 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 505 ms to try 5000 different iterations.  Found a solution that moves 2560 regions; Going from a computed imbalance of 0.17576569542457532 to a new imbalance of 0.015504407375549343. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022513245882803602); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03462805412430022, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.28444444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022513245882803602); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012166666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04741666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-19 12:54:01,836 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv552030613:73 , srv1240406676:79 , srv1298241979:79 , srv132851632:80 , srv1390972464:80 , srv1046137911:81 , srv969513038:81 , srv1009921656:82 , srv103830899:82 , srv1186878549:82 , srv1261719314:82 , srv1976628776:82 , srv396896953:82 , srv725461257:82 , srv1023756996:83 , srv1207701725:83 , srv1445241560:83 , srv1468433128:83 , srv188657073:83 , srv1956549818:83 , srv2129024947:83 , srv2140601606:83 , srv633318387:83 , srv678409492:83 , srv724186438:83 , srv793603965:83 , srv992319801:83 , srv1075036867:84 , srv1567225327:84 , srv2039048773:84 , srv2083323731:84 , srv233859433:84 , srv468994665:84 , srv562664628:84 , srv61853811:84 , srv650335536:84 , srv788900359:84 , srv933015772:84 , srv114799786:85 , srv1273173732:85 , srv140895142:85 , srv1624254264:85 , srv2041030926:85 , srv2060876183:85 , srv768340453:85 , srv98143725:85 , srv994996126:85 , srv1016919167:86 , srv1017732837:86 , srv1376742936:86 , srv1611297795:86 , srv1637089338:86 , srv2138145618:86 , srv261363240:86 , srv27263691:86 , srv374288556:86 , srv937506904:86 , srv1059252006:87 , srv1218995925:87 , srv1300339401:87 , srv1366860215:87 , srv1429261044:87 , srv1441522114:87 , srv1595086038:87 , srv1637040790:87 , srv1863002799:87 , srv1935331893:87 , srv438221467:87 , srv562862418:87 , srv626568787:87 , srv678997394:87 , srv9106584:87 , srv1436922347:88 , srv1518418677:88 , srv2045289259:88 , srv2098740511:88 , srv2121622720:88 , srv212659528:88 , srv41596212:88 , srv979260650:88 , srv102353437:89 , srv1366595432:89 , srv1528452956:89 , srv193237065:89 , srv232690724:89 , srv530976730:89 , srv1275683056:90 , srv1585204045:90 , srv284243024:90 , srv577871485:90 , srv1211273243:91 , srv446472230:91 , srv966770169:91 , srv1311866145:92 , srv1491198008:92 , srv2019977164:92 , srv682558210:92 , srv573365634:93 , srv936233759:94 , srv362182774:514 }
2023-02-19 12:54:01,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-19 12:54:01,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv562862418=75, srv1935331893=45, srv362182774=65, srv212659528=56, srv132851632=21, srv1518418677=33, srv232690724=60, srv2140601606=59, srv793603965=89, srv1390972464=25, srv261363240=62, srv1298241979=18, srv678409492=82, srv374288556=66, srv1075036867=8, srv1275683056=17, srv193237065=44, srv979260650=96, srv1023756996=4, srv933015772=91, srv61853811=78, srv552030613=73, srv1311866145=20, srv1637040790=40, srv98143725=97, srv1211273243=12, srv446472230=70, srv1366595432=22, srv396896953=67, srv1445241560=30, srv626568787=79, srv1567225327=35, srv41596212=68, srv1436922347=28, srv1585204045=36, srv1976628776=47, srv2098740511=54, srv788900359=88, srv682558210=84, srv102353437=3, srv2083323731=53, srv1595086038=37, srv1009921656=0, srv1468433128=31, srv1624254264=39, srv633318387=80, srv1261719314=15, srv936233759=92, srv1207701725=11, srv1300339401=19, srv1429261044=27, srv140895142=26, srv678997394=83, srv969513038=95, srv650335536=81, srv1366860215=23, srv994996126=99, srv2045289259=51, srv468994665=71, srv1218995925=13, srv2019977164=48, srv966770169=94, srv725461257=86, srv1637089338=41, srv284243024=64, srv724186438=85, srv992319801=98, srv1017732837=2, srv2129024947=57, srv937506904=93, srv233859433=61, srv1186878549=10, srv573365634=76, srv1956549818=46, srv530976730=72, srv103830899=5, srv1611297795=38, srv2121622720=55, srv27263691=63, srv1016919167=1, srv188657073=43, srv1441522114=29, srv562664628=74, srv2060876183=52, srv1273173732=16, srv2138145618=58, srv9106584=90, srv1240406676=14, srv1046137911=6, srv2041030926=50, srv1863002799=42, srv768340453=87, srv1376742936=24, srv438221467=69, srv577871485=77, srv114799786=9, srv2039048773=49, srv1059252006=7, srv1528452956=34, srv1491198008=32} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-19 12:54:01,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-19 12:54:01,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-19 12:54:01,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-19 12:54:01,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-19 12:54:01,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-19 12:54:01,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-19 12:54:01,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-19 12:54:01,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-19 12:54:01,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-19 12:54:01,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-19 12:54:01,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-19 12:54:01,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-19 12:54:01,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-19 12:54:01,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-19 12:54:01,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-19 12:54:01,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-19 12:54:01,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-19 12:54:01,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-19 12:54:01,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-19 12:54:01,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-19 12:54:01,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-19 12:54:01,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-19 12:54:01,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-19 12:54:01,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-19 12:54:01,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-19 12:54:01,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-19 12:54:01,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-19 12:54:01,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-19 12:54:01,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-19 12:54:01,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-19 12:54:01,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-19 12:54:01,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-19 12:54:01,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-19 12:54:01,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-19 12:54:01,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-19 12:54:01,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-19 12:54:01,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-19 12:54:01,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-19 12:54:01,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-19 12:54:01,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-19 12:54:01,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-19 12:54:01,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-19 12:54:01,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-19 12:54:01,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-19 12:54:01,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-19 12:54:01,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-19 12:54:01,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-19 12:54:01,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-19 12:54:01,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-19 12:54:01,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-19 12:54:01,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-19 12:54:01,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-19 12:54:01,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-19 12:54:01,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-19 12:54:01,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-19 12:54:01,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-19 12:54:01,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-19 12:54:01,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-19 12:54:01,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-19 12:54:01,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-19 12:54:01,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-19 12:54:01,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-19 12:54:01,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-19 12:54:01,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-19 12:54:01,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-19 12:54:01,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-19 12:54:01,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-19 12:54:01,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-19 12:54:01,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-19 12:54:01,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-19 12:54:01,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-19 12:54:01,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-19 12:54:01,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-19 12:54:01,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-19 12:54:01,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-19 12:54:01,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-19 12:54:01,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-19 12:54:01,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-19 12:54:01,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-19 12:54:01,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-19 12:54:01,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-19 12:54:01,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-19 12:54:01,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-19 12:54:01,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-19 12:54:01,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-19 12:54:01,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-19 12:54:01,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-19 12:54:01,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-19 12:54:01,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-19 12:54:01,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-19 12:54:01,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-19 12:54:01,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-19 12:54:01,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-19 12:54:01,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-19 12:54:01,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-19 12:54:01,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-19 12:54:01,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-19 12:54:01,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-19 12:54:01,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-19 12:54:01,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-19 12:54:01,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-19 12:54:02,097 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022513245882803602); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03462805412430022, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022513245882803602); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012166666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04741666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-19 12:54:02,098 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-19 12:54:02,098 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.015486481872419883, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022513245882803602); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03462805412430022, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022513245882803602); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012166666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04741666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-19 12:54:02,410 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 508 ms to try 5000 different iterations.  Found a solution that moves 1016 regions; Going from a computed imbalance of 0.015486481872419883 to a new imbalance of 7.131596216788697E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006344635199123064); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008395391820274807); MoveCostFunction : (multiplier=7.0, imbalance=0.11288888888888889, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006344635199123064); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004583333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-19 12:54:02,413 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1311866145:82 , srv1518418677:82 , srv2019977164:82 , srv1366860215:83 , srv1528452956:83 , srv193237065:83 , srv2098740511:83 , srv573365634:83 , srv626568787:83 , srv936233759:83 , srv1016919167:84 , srv102353437:84 , srv1218995925:84 , srv1273173732:84 , srv1390972464:84 , srv1491198008:84 , srv1595086038:84 , srv1624254264:84 , srv1637040790:84 , srv1637089338:84 , srv1863002799:84 , srv2045289259:84 , srv2129024947:84 , srv2140601606:84 , srv233859433:84 , srv41596212:84 , srv446472230:84 , srv530976730:84 , srv577871485:84 , srv725461257:84 , srv9106584:84 , srv1023756996:85 , srv103830899:85 , srv1275683056:85 , srv132851632:85 , srv1366595432:85 , srv1441522114:85 , srv1567225327:85 , srv2041030926:85 , srv212659528:85 , srv232690724:85 , srv284243024:85 , srv438221467:85 , srv468994665:85 , srv562862418:85 , srv61853811:85 , srv633318387:85 , srv650335536:85 , srv678409492:85 , srv682558210:85 , srv724186438:85 , srv966770169:85 , srv979260650:85 , srv1017732837:86 , srv1059252006:86 , srv114799786:86 , srv1207701725:86 , srv1211273243:86 , srv1298241979:86 , srv1300339401:86 , srv1376742936:86 , srv140895142:86 , srv1429261044:86 , srv1585204045:86 , srv1935331893:86 , srv1956549818:86 , srv2060876183:86 , srv2121622720:86 , srv562664628:86 , srv768340453:86 , srv933015772:86 , srv937506904:86 , srv992319801:86 , srv1046137911:87 , srv1075036867:87 , srv1240406676:87 , srv1436922347:87 , srv1468433128:87 , srv1611297795:87 , srv188657073:87 , srv1976628776:87 , srv2083323731:87 , srv2138145618:87 , srv27263691:87 , srv396896953:87 , srv678997394:87 , srv969513038:87 , srv1186878549:88 , srv1261719314:88 , srv261363240:88 , srv374288556:88 , srv552030613:88 , srv788900359:88 , srv793603965:88 , srv98143725:88 , srv994996126:88 , srv1445241560:89 , srv1009921656:90 , srv2039048773:90 , srv362182774:540 }
2023-02-19 12:54:02,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-19 12:54:02,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv562862418=75, srv1935331893=45, srv362182774=65, srv212659528=56, srv132851632=21, srv1518418677=33, srv232690724=60, srv2140601606=59, srv793603965=89, srv1390972464=25, srv261363240=62, srv1298241979=18, srv678409492=82, srv374288556=66, srv1075036867=8, srv1275683056=17, srv193237065=44, srv979260650=96, srv1023756996=4, srv933015772=91, srv61853811=78, srv552030613=73, srv1311866145=20, srv1637040790=40, srv98143725=97, srv1211273243=12, srv446472230=70, srv1366595432=22, srv396896953=67, srv1445241560=30, srv626568787=79, srv1567225327=35, srv41596212=68, srv1436922347=28, srv1585204045=36, srv1976628776=47, srv2098740511=54, srv788900359=88, srv682558210=84, srv102353437=3, srv2083323731=53, srv1595086038=37, srv1009921656=0, srv1468433128=31, srv1624254264=39, srv633318387=80, srv1261719314=15, srv936233759=92, srv1207701725=11, srv1300339401=19, srv1429261044=27, srv140895142=26, srv678997394=83, srv969513038=95, srv650335536=81, srv1366860215=23, srv994996126=99, srv2045289259=51, srv468994665=71, srv1218995925=13, srv2019977164=48, srv966770169=94, srv725461257=86, srv1637089338=41, srv284243024=64, srv724186438=85, srv992319801=98, srv1017732837=2, srv2129024947=57, srv937506904=93, srv233859433=61, srv1186878549=10, srv573365634=76, srv1956549818=46, srv530976730=72, srv103830899=5, srv1611297795=38, srv2121622720=55, srv27263691=63, srv1016919167=1, srv188657073=43, srv1441522114=29, srv562664628=74, srv2060876183=52, srv1273173732=16, srv2138145618=58, srv9106584=90, srv1240406676=14, srv1046137911=6, srv2041030926=50, srv1863002799=42, srv768340453=87, srv1376742936=24, srv438221467=69, srv577871485=77, srv114799786=9, srv2039048773=49, srv1059252006=7, srv1528452956=34, srv1491198008=32} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-19 12:54:02,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-19 12:54:02,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-19 12:54:02,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-19 12:54:02,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-19 12:54:02,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-19 12:54:02,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-19 12:54:02,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-19 12:54:02,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-19 12:54:02,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-19 12:54:02,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-19 12:54:02,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-19 12:54:02,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-19 12:54:02,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-19 12:54:02,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-19 12:54:02,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-19 12:54:02,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-19 12:54:02,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-19 12:54:02,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-19 12:54:02,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-19 12:54:02,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-19 12:54:02,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-19 12:54:02,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-19 12:54:02,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-19 12:54:02,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-19 12:54:02,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-19 12:54:02,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-19 12:54:02,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-19 12:54:02,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-19 12:54:02,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-19 12:54:02,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-19 12:54:02,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-19 12:54:02,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-19 12:54:02,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-19 12:54:02,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-19 12:54:02,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-19 12:54:02,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-19 12:54:02,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-19 12:54:02,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-19 12:54:02,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-19 12:54:02,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-19 12:54:02,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-19 12:54:02,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-19 12:54:02,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-19 12:54:02,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-19 12:54:02,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-19 12:54:02,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-19 12:54:02,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-19 12:54:02,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-19 12:54:02,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-19 12:54:02,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-19 12:54:02,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-19 12:54:02,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-19 12:54:02,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-19 12:54:02,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-19 12:54:02,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-19 12:54:02,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-19 12:54:02,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-19 12:54:02,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-19 12:54:02,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-19 12:54:02,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-19 12:54:02,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-19 12:54:02,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-19 12:54:02,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-19 12:54:02,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-19 12:54:02,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-19 12:54:02,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-19 12:54:02,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-19 12:54:02,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-19 12:54:02,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-19 12:54:02,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-19 12:54:02,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-19 12:54:02,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-19 12:54:02,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-19 12:54:02,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-19 12:54:02,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-19 12:54:02,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-19 12:54:02,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-19 12:54:02,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-19 12:54:02,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-19 12:54:02,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-19 12:54:02,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-19 12:54:02,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-19 12:54:02,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-19 12:54:02,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-19 12:54:02,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-19 12:54:02,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-19 12:54:02,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-19 12:54:02,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-19 12:54:02,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-19 12:54:02,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-19 12:54:02,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-19 12:54:02,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-19 12:54:02,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-19 12:54:02,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-19 12:54:02,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-19 12:54:02,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-19 12:54:02,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-19 12:54:02,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-19 12:54:02,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-19 12:54:02,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-19 12:54:02,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-19 12:54:02,552 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006344635199123064); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008395391820274807); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006344635199123064); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004583333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-19 12:54:02,552 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-19 12:54:02,552 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=7.060454376243648E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006344635199123064); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008395391820274807); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006344635199123064); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004583333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-19 12:54:02,899 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 443 ms to try 5000 different iterations.  Found a solution that moves 275 regions; Going from a computed imbalance of 7.060454376243648E-4 to a new imbalance of 2.062745389033195E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.030555555555555555, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-19 12:54:02,900 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv2039048773:80 , srv1436922347:83 , srv1468433128:83 , srv2083323731:83 , srv793603965:83 , srv98143725:83 , srv994996126:83 , srv1009921656:84 , srv1016919167:84 , srv1023756996:84 , srv1059252006:84 , srv114799786:84 , srv1186878549:84 , srv1218995925:84 , srv1240406676:84 , srv1275683056:84 , srv1366595432:84 , srv1441522114:84 , srv1518418677:84 , srv1567225327:84 , srv1585204045:84 , srv1595086038:84 , srv1637089338:84 , srv1956549818:84 , srv2060876183:84 , srv212659528:84 , srv232690724:84 , srv261363240:84 , srv27263691:84 , srv284243024:84 , srv374288556:84 , srv61853811:84 , srv626568787:84 , srv633318387:84 , srv678409492:84 , srv936233759:84 , srv937506904:84 , srv966770169:84 , srv969513038:84 , srv1017732837:85 , srv102353437:85 , srv103830899:85 , srv1046137911:85 , srv1075036867:85 , srv1207701725:85 , srv1211273243:85 , srv1261719314:85 , srv1273173732:85 , srv1298241979:85 , srv1300339401:85 , srv132851632:85 , srv1366860215:85 , srv1376742936:85 , srv1390972464:85 , srv140895142:85 , srv1429261044:85 , srv1445241560:85 , srv1491198008:85 , srv1528452956:85 , srv1611297795:85 , srv1624254264:85 , srv1863002799:85 , srv188657073:85 , srv193237065:85 , srv1935331893:85 , srv1976628776:85 , srv2019977164:85 , srv2041030926:85 , srv2045289259:85 , srv2098740511:85 , srv2121622720:85 , srv2129024947:85 , srv2138145618:85 , srv2140601606:85 , srv233859433:85 , srv396896953:85 , srv41596212:85 , srv438221467:85 , srv446472230:85 , srv468994665:85 , srv530976730:85 , srv552030613:85 , srv562664628:85 , srv562862418:85 , srv573365634:85 , srv650335536:85 , srv678997394:85 , srv682558210:85 , srv724186438:85 , srv725461257:85 , srv768340453:85 , srv788900359:85 , srv9106584:85 , srv933015772:85 , srv979260650:85 , srv1311866145:86 , srv577871485:86 , srv992319801:86 , srv1637040790:87 , srv362182774:629 }
2023-02-19 12:54:02,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-19 12:54:02,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv562862418=75, srv1935331893=45, srv362182774=65, srv212659528=56, srv132851632=21, srv1518418677=33, srv232690724=60, srv2140601606=59, srv793603965=89, srv1390972464=25, srv261363240=62, srv1298241979=18, srv678409492=82, srv374288556=66, srv1075036867=8, srv1275683056=17, srv193237065=44, srv979260650=96, srv1023756996=4, srv933015772=91, srv61853811=78, srv552030613=73, srv1311866145=20, srv1637040790=40, srv98143725=97, srv1211273243=12, srv446472230=70, srv1366595432=22, srv396896953=67, srv1445241560=30, srv626568787=79, srv1567225327=35, srv41596212=68, srv1436922347=28, srv1585204045=36, srv1976628776=47, srv2098740511=54, srv788900359=88, srv682558210=84, srv102353437=3, srv2083323731=53, srv1595086038=37, srv1009921656=0, srv1468433128=31, srv1624254264=39, srv633318387=80, srv1261719314=15, srv936233759=92, srv1207701725=11, srv1300339401=19, srv1429261044=27, srv140895142=26, srv678997394=83, srv969513038=95, srv650335536=81, srv1366860215=23, srv994996126=99, srv2045289259=51, srv468994665=71, srv1218995925=13, srv2019977164=48, srv966770169=94, srv725461257=86, srv1637089338=41, srv284243024=64, srv724186438=85, srv992319801=98, srv1017732837=2, srv2129024947=57, srv937506904=93, srv233859433=61, srv1186878549=10, srv573365634=76, srv1956549818=46, srv530976730=72, srv103830899=5, srv1611297795=38, srv2121622720=55, srv27263691=63, srv1016919167=1, srv188657073=43, srv1441522114=29, srv562664628=74, srv2060876183=52, srv1273173732=16, srv2138145618=58, srv9106584=90, srv1240406676=14, srv1046137911=6, srv2041030926=50, srv1863002799=42, srv768340453=87, srv1376742936=24, srv438221467=69, srv577871485=77, srv114799786=9, srv2039048773=49, srv1059252006=7, srv1528452956=34, srv1491198008=32} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-19 12:54:02,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-19 12:54:02,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-19 12:54:02,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-19 12:54:02,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-19 12:54:02,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-19 12:54:02,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-19 12:54:02,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-19 12:54:02,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-19 12:54:02,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-19 12:54:02,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-19 12:54:02,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-19 12:54:02,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-19 12:54:02,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-19 12:54:02,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-19 12:54:02,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-19 12:54:02,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-19 12:54:02,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-19 12:54:02,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-19 12:54:02,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-19 12:54:02,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-19 12:54:02,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-19 12:54:02,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-19 12:54:02,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-19 12:54:02,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-19 12:54:02,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-19 12:54:02,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-19 12:54:02,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-19 12:54:02,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-19 12:54:02,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-19 12:54:02,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-19 12:54:02,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-19 12:54:02,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-19 12:54:02,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-19 12:54:02,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-19 12:54:02,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-19 12:54:02,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-19 12:54:02,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-19 12:54:02,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-19 12:54:02,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-19 12:54:02,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-19 12:54:02,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-19 12:54:02,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-19 12:54:02,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-19 12:54:02,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-19 12:54:02,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-19 12:54:02,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-19 12:54:02,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-19 12:54:02,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-19 12:54:02,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-19 12:54:02,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-19 12:54:02,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-19 12:54:02,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-19 12:54:02,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-19 12:54:02,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-19 12:54:02,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-19 12:54:02,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-19 12:54:02,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-19 12:54:02,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-19 12:54:02,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-19 12:54:02,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-19 12:54:02,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-19 12:54:02,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-19 12:54:02,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-19 12:54:02,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-19 12:54:02,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-19 12:54:02,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-19 12:54:02,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-19 12:54:02,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-19 12:54:02,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-19 12:54:02,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-19 12:54:02,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-19 12:54:02,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-19 12:54:02,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-19 12:54:02,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-19 12:54:02,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-19 12:54:02,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-19 12:54:02,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-19 12:54:02,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-19 12:54:02,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-19 12:54:02,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-19 12:54:02,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-19 12:54:02,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-19 12:54:02,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-19 12:54:02,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-19 12:54:02,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-19 12:54:02,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-19 12:54:02,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-19 12:54:02,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-19 12:54:02,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-19 12:54:02,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-19 12:54:02,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-19 12:54:02,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-19 12:54:02,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-19 12:54:02,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-19 12:54:02,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-19 12:54:02,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-19 12:54:02,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-19 12:54:02,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-19 12:54:02,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-19 12:54:02,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-19 12:54:02,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-19 12:54:03,072 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.8701862733846913E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-19 12:54:03,072 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv2039048773:80 , srv1436922347:83 , srv1468433128:83 , srv2083323731:83 , srv793603965:83 , srv98143725:83 , srv994996126:83 , srv1009921656:84 , srv1016919167:84 , srv1023756996:84 , srv1059252006:84 , srv114799786:84 , srv1186878549:84 , srv1218995925:84 , srv1240406676:84 , srv1275683056:84 , srv1366595432:84 , srv1441522114:84 , srv1518418677:84 , srv1567225327:84 , srv1585204045:84 , srv1595086038:84 , srv1637089338:84 , srv1956549818:84 , srv2060876183:84 , srv212659528:84 , srv232690724:84 , srv261363240:84 , srv27263691:84 , srv284243024:84 , srv374288556:84 , srv61853811:84 , srv626568787:84 , srv633318387:84 , srv678409492:84 , srv936233759:84 , srv937506904:84 , srv966770169:84 , srv969513038:84 , srv1017732837:85 , srv102353437:85 , srv103830899:85 , srv1046137911:85 , srv1075036867:85 , srv1207701725:85 , srv1211273243:85 , srv1261719314:85 , srv1273173732:85 , srv1298241979:85 , srv1300339401:85 , srv132851632:85 , srv1366860215:85 , srv1376742936:85 , srv1390972464:85 , srv140895142:85 , srv1429261044:85 , srv1445241560:85 , srv1491198008:85 , srv1528452956:85 , srv1611297795:85 , srv1624254264:85 , srv1863002799:85 , srv188657073:85 , srv193237065:85 , srv1935331893:85 , srv1976628776:85 , srv2019977164:85 , srv2041030926:85 , srv2045289259:85 , srv2098740511:85 , srv2121622720:85 , srv2129024947:85 , srv2138145618:85 , srv2140601606:85 , srv233859433:85 , srv396896953:85 , srv41596212:85 , srv438221467:85 , srv446472230:85 , srv468994665:85 , srv530976730:85 , srv552030613:85 , srv562664628:85 , srv562862418:85 , srv573365634:85 , srv650335536:85 , srv678997394:85 , srv682558210:85 , srv724186438:85 , srv725461257:85 , srv768340453:85 , srv788900359:85 , srv9106584:85 , srv933015772:85 , srv979260650:85 , srv1311866145:86 , srv577871485:86 , srv992319801:86 , srv1637040790:87 , srv362182774:629 }
2023-02-19 12:54:03,108 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=325 (was 325), ProcessCount=170 (was 170), AvailableMemoryMB=10196 (was 10703)
2023-02-19 12:54:03,119 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=325, ProcessCount=170, AvailableMemoryMB=10195
2023-02-19 12:54:03,120 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-19 12:54:03,120 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-02-19 12:54:03,133 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1005018974:84 , srv1059963872:84 , srv1128650687:84 , srv114000377:84 , srv12440534:84 , srv125863029:84 , srv129571186:84 , srv1305197217:84 , srv1356400068:84 , srv1359245524:84 , srv1399544822:84 , srv1547823971:84 , srv1551092978:84 , srv1704215563:84 , srv1731098672:84 , srv1842050629:84 , srv1940879698:84 , srv1962556344:84 , srv2021775521:84 , srv2049343206:84 , srv2137719719:84 , srv371157062:84 , srv419844965:84 , srv545000918:84 , srv64646545:84 , srv714311399:84 , srv786054781:84 , srv806359606:84 , srv86319602:84 , srv1255008670:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-02-19 12:54:03,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-19 12:54:03,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1356400068=9, srv12440534=4, srv419844965=23, srv714311399=26, srv1399544822=11, srv1704215563=14, srv129571186=7, srv1551092978=13, srv1305197217=8, srv2049343206=20, srv1940879698=17, srv86319602=29, srv371157062=22, srv1359245524=10, srv806359606=28, srv1962556344=18, srv125863029=6, srv2021775521=19, srv1005018974=0, srv64646545=25, srv1128650687=2, srv1842050629=16, srv1731098672=15, srv1059963872=1, srv786054781=27, srv1547823971=12, srv1255008670=5, srv2137719719=21, srv114000377=3, srv545000918=24} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-19 12:54:03,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-19 12:54:03,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-19 12:54:03,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-19 12:54:03,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-19 12:54:03,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-19 12:54:03,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-19 12:54:03,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-19 12:54:03,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-19 12:54:03,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-19 12:54:03,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-19 12:54:03,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-19 12:54:03,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-19 12:54:03,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-19 12:54:03,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-19 12:54:03,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-19 12:54:03,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-19 12:54:03,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-19 12:54:03,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-19 12:54:03,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-19 12:54:03,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-19 12:54:03,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-19 12:54:03,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-19 12:54:03,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-19 12:54:03,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-19 12:54:03,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-19 12:54:03,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-19 12:54:03,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-19 12:54:03,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-19 12:54:03,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-19 12:54:03,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-19 12:54:03,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-02-19 12:54:03,175 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1666014916078837, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-19 12:54:03,175 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912717999124295, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1666014916078837, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-02-19 12:56:03,156 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5997947 different iterations.  Found a solution that moves 1860 regions; Going from a computed imbalance of 0.9912717999124295 to a new imbalance of 4.341332789166274E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6888888888888889, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-19 12:56:03,161 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1005018974:90 , srv1059963872:90 , srv1128650687:90 , srv114000377:90 , srv12440534:90 , srv1255008670:90 , srv125863029:90 , srv129571186:90 , srv1305197217:90 , srv1356400068:90 , srv1359245524:90 , srv1399544822:90 , srv1547823971:90 , srv1551092978:90 , srv1704215563:90 , srv1731098672:90 , srv1842050629:90 , srv1940879698:90 , srv1962556344:90 , srv2021775521:90 , srv2049343206:90 , srv2137719719:90 , srv371157062:90 , srv419844965:90 , srv545000918:90 , srv64646545:90 , srv714311399:90 , srv786054781:90 , srv806359606:90 , srv86319602:90 }
2023-02-19 12:56:03,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-19 12:56:03,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1356400068=9, srv12440534=4, srv419844965=23, srv714311399=26, srv1399544822=11, srv1704215563=14, srv129571186=7, srv1551092978=13, srv1305197217=8, srv2049343206=20, srv1940879698=17, srv86319602=29, srv371157062=22, srv1359245524=10, srv806359606=28, srv1962556344=18, srv125863029=6, srv2021775521=19, srv1005018974=0, srv64646545=25, srv1128650687=2, srv1842050629=16, srv1731098672=15, srv1059963872=1, srv786054781=27, srv1547823971=12, srv1255008670=5, srv2137719719=21, srv114000377=3, srv545000918=24} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-19 12:56:03,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-19 12:56:03,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-19 12:56:03,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-19 12:56:03,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-19 12:56:03,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-19 12:56:03,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-19 12:56:03,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-19 12:56:03,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-19 12:56:03,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-19 12:56:03,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-19 12:56:03,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-19 12:56:03,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-19 12:56:03,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-19 12:56:03,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-19 12:56:03,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-19 12:56:03,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-19 12:56:03,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-19 12:56:03,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-19 12:56:03,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-19 12:56:03,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-19 12:56:03,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-19 12:56:03,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-19 12:56:03,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-19 12:56:03,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-19 12:56:03,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-19 12:56:03,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-19 12:56:03,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-19 12:56:03,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-19 12:56:03,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-19 12:56:03,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-19 12:56:03,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-02-19 12:56:03,179 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-19 12:56:03,179 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1005018974:90 , srv1059963872:90 , srv1128650687:90 , srv114000377:90 , srv12440534:90 , srv1255008670:90 , srv125863029:90 , srv129571186:90 , srv1305197217:90 , srv1356400068:90 , srv1359245524:90 , srv1399544822:90 , srv1547823971:90 , srv1551092978:90 , srv1704215563:90 , srv1731098672:90 , srv1842050629:90 , srv1940879698:90 , srv1962556344:90 , srv2021775521:90 , srv2049343206:90 , srv2137719719:90 , srv371157062:90 , srv419844965:90 , srv545000918:90 , srv64646545:90 , srv714311399:90 , srv786054781:90 , srv806359606:90 , srv86319602:90 }
2023-02-19 12:56:03,192 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=200 (was 325), ProcessCount=170 (was 170), AvailableMemoryMB=10014 (was 10195)