2023-03-29 03:55:55,679 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-03-29 03:55:56,063 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-03-29 03:55:56,191 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-03-29 03:55:56,267 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-29 03:55:56,278 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-03-29 03:55:56,295 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=409, ProcessCount=169, AvailableMemoryMB=7532
2023-03-29 03:55:56,297 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-29 03:55:56,298 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-03-29 03:55:56,453 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1013573134:84 , srv102864432:84 , srv1057320898:84 , srv106169244:84 , srv1092136557:84 , srv1094039387:84 , srv1114850048:84 , srv1148243188:84 , srv1165718431:84 , srv1179671223:84 , srv1185104802:84 , srv1215111640:84 , srv1216564897:84 , srv1261791637:84 , srv1289496794:84 , srv1318975988:84 , srv1333682714:84 , srv1340670106:84 , srv1395681284:84 , srv1403704598:84 , srv1429048956:84 , srv1440887355:84 , srv1462625755:84 , srv1492621600:84 , srv1507447850:84 , srv1570673706:84 , srv1610468437:84 , srv1610575862:84 , srv1690592292:84 , srv172537197:84 , srv1742489991:84 , srv1781861026:84 , srv1783768378:84 , srv1786642941:84 , srv1790039759:84 , srv1801915109:84 , srv1862263035:84 , srv1866558889:84 , srv1866825677:84 , srv1873764958:84 , srv1893844518:84 , srv1925726655:84 , srv1930739750:84 , srv1955970091:84 , srv1956078712:84 , srv1956999066:84 , srv1979272760:84 , srv1988596505:84 , srv1989360321:84 , srv20479490:84 , srv2094908416:84 , srv2098775844:84 , srv2123983890:84 , srv2126200699:84 , srv233110714:84 , srv241751107:84 , srv286666855:84 , srv294209228:84 , srv351497537:84 , srv355373880:84 , srv37956910:84 , srv381018484:84 , srv381038534:84 , srv44310704:84 , srv453567040:84 , srv46522739:84 , srv46717968:84 , srv481812578:84 , srv507115897:84 , srv514986774:84 , srv522564382:84 , srv531050543:84 , srv595962734:84 , srv598216748:84 , srv620722305:84 , srv630969112:84 , srv727707620:84 , srv729881032:84 , srv739896597:84 , srv751121962:84 , srv751764623:84 , srv76284902:84 , srv76649140:84 , srv781544735:84 , srv802250739:84 , srv817256657:84 , srv823098907:84 , srv831751529:84 , srv847711571:84 , srv864833024:84 , srv87473540:84 , srv875184987:84 , srv883328781:84 , srv887808902:84 , srv927666158:84 , srv933289332:84 , srv948546288:84 , srv969421253:84 , srv978326568:84 , srv1652732052:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-03-29 03:55:56,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-29 03:55:56,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1215111640=11, srv514986774=70, srv46717968=67, srv831751529=88, srv864833024=90, srv37956910=61, srv102864432=1, srv1148243188=7, srv1873764958=40, srv46522739=66, srv453567040=65, srv1013573134=0, srv751121962=80, srv1925726655=42, srv1781861026=32, srv481812578=68, srv1507447850=24, srv1395681284=18, srv522564382=71, srv875184987=92, srv1956999066=46, srv933289332=96, srv1492621600=23, srv531050543=72, srv1057320898=2, srv1652732052=28, srv1989360321=49, srv76284902=82, srv1165718431=8, srv1866558889=38, srv1462625755=22, srv241751107=56, srv927666158=95, srv1955970091=44, srv727707620=77, srv1185104802=10, srv1930739750=43, srv1179671223=9, srv44310704=64, srv2094908416=51, srv1289496794=14, srv781544735=84, srv1570673706=25, srv847711571=89, srv1216564897=12, srv172537197=30, srv823098907=87, srv294209228=58, srv1979272760=47, srv598216748=74, srv883328781=93, srv106169244=3, srv1261791637=13, srv1786642941=34, srv751764623=81, srv1092136557=4, srv87473540=91, srv1801915109=36, srv1862263035=37, srv2098775844=52, srv1783768378=33, srv1440887355=21, srv969421253=98, srv1893844518=41, srv355373880=60, srv817256657=86, srv1094039387=5, srv1429048956=20, srv381038534=63, srv351497537=59, srv1866825677=39, srv739896597=79, srv978326568=99, srv1333682714=16, srv1790039759=35, srv948546288=97, srv1956078712=45, srv1114850048=6, srv507115897=69, srv1610575862=27, srv729881032=78, srv2123983890=53, srv1403704598=19, srv1742489991=31, srv20479490=50, srv286666855=57, srv630969112=76, srv76649140=83, srv1340670106=17, srv887808902=94, srv381018484=62, srv802250739=85, srv1690592292=29, srv1610468437=26, srv1318975988=15, srv595962734=73, srv2126200699=54, srv233110714=55, srv620722305=75, srv1988596505=48} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-29 03:55:56,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-29 03:55:56,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-29 03:55:56,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-29 03:55:56,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-29 03:55:56,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-29 03:55:56,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-29 03:55:56,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-29 03:55:56,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-29 03:55:56,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-29 03:55:56,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-29 03:55:56,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-29 03:55:56,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-29 03:55:56,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-29 03:55:56,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-29 03:55:56,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-29 03:55:56,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-29 03:55:56,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-29 03:55:56,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-29 03:55:56,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-29 03:55:56,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-29 03:55:56,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-29 03:55:56,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-29 03:55:56,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-29 03:55:56,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-29 03:55:56,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-29 03:55:56,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-29 03:55:56,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-29 03:55:56,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-29 03:55:56,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-29 03:55:56,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-29 03:55:56,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-29 03:55:56,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-29 03:55:56,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-29 03:55:56,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-29 03:55:56,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-29 03:55:56,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-29 03:55:56,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-29 03:55:56,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-29 03:55:56,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-29 03:55:56,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-29 03:55:56,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-29 03:55:56,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-29 03:55:56,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-29 03:55:56,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-29 03:55:56,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-29 03:55:56,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-29 03:55:56,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-29 03:55:56,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-29 03:55:56,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-29 03:55:56,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-29 03:55:56,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-29 03:55:56,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-29 03:55:56,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-29 03:55:56,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-29 03:55:56,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-29 03:55:56,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-29 03:55:56,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-29 03:55:56,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-29 03:55:56,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-29 03:55:56,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-29 03:55:56,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-29 03:55:56,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-29 03:55:56,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-29 03:55:56,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-29 03:55:56,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-29 03:55:56,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-29 03:55:56,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-29 03:55:56,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-29 03:55:56,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-29 03:55:56,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-29 03:55:56,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-29 03:55:56,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-29 03:55:56,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-29 03:55:56,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-29 03:55:56,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-29 03:55:56,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-29 03:55:56,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-29 03:55:56,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-29 03:55:56,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-29 03:55:56,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-29 03:55:56,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-29 03:55:56,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-29 03:55:56,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-29 03:55:56,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-29 03:55:56,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-29 03:55:56,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-29 03:55:56,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-29 03:55:56,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-29 03:55:56,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-29 03:55:56,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-29 03:55:56,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-29 03:55:56,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-29 03:55:56,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-29 03:55:56,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-29 03:55:56,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-29 03:55:56,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-29 03:55:56,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-29 03:55:56,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-29 03:55:56,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-29 03:55:56,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-29 03:55:56,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-29 03:55:56,835 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-29 03:55:56,835 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-29 03:55:56,836 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-29 03:55:57,199 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 572 ms to try 5000 different iterations.  Found a solution that moves 4645 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18390480702911252. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04127413305097533, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05213295615870173, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5161111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04127413305097533, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.179, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24758333333333332, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-29 03:55:57,240 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv620722305:82 , srv1462625755:83 , srv1610468437:83 , srv1989360321:83 , srv2126200699:83 , srv598216748:83 , srv751121962:83 , srv76649140:83 , srv87473540:83 , srv102864432:84 , srv1148243188:84 , srv1215111640:84 , srv1216564897:84 , srv1261791637:84 , srv1440887355:84 , srv1783768378:84 , srv1893844518:84 , srv1956999066:84 , srv2094908416:84 , srv294209228:84 , srv453567040:84 , srv481812578:84 , srv751764623:84 , srv76284902:84 , srv1057320898:85 , srv1092136557:85 , srv1114850048:85 , srv1318975988:85 , srv1403704598:85 , srv1492621600:85 , srv1610575862:85 , srv381018484:85 , srv727707620:85 , srv817256657:85 , srv823098907:85 , srv847711571:85 , srv864833024:85 , srv1094039387:86 , srv1165718431:86 , srv1289496794:86 , srv1570673706:86 , srv1742489991:86 , srv1862263035:86 , srv1866558889:86 , srv1866825677:86 , srv1873764958:86 , srv1925726655:86 , srv241751107:86 , srv286666855:86 , srv37956910:86 , srv46522739:86 , srv46717968:86 , srv522564382:86 , srv531050543:86 , srv729881032:86 , srv875184987:86 , srv927666158:86 , srv978326568:86 , srv1013573134:87 , srv106169244:87 , srv1179671223:87 , srv1340670106:87 , srv1429048956:87 , srv1690592292:87 , srv1781861026:87 , srv1790039759:87 , srv1956078712:87 , srv233110714:87 , srv351497537:87 , srv44310704:87 , srv507115897:87 , srv514986774:87 , srv739896597:87 , srv802250739:87 , srv883328781:87 , srv887808902:87 , srv948546288:87 , srv1979272760:88 , srv20479490:88 , srv2098775844:88 , srv595962734:88 , srv630969112:88 , srv933289332:88 , srv969421253:88 , srv1185104802:89 , srv1395681284:89 , srv172537197:89 , srv1801915109:89 , srv1930739750:89 , srv2123983890:89 , srv381038534:89 , srv781544735:89 , srv355373880:90 , srv1786642941:91 , srv831751529:91 , srv1507447850:92 , srv1955970091:92 , srv1988596505:93 , srv1333682714:96 , srv1652732052:457 }
2023-03-29 03:55:57,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-29 03:55:57,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1215111640=11, srv514986774=70, srv46717968=67, srv831751529=88, srv864833024=90, srv37956910=61, srv102864432=1, srv1148243188=7, srv1873764958=40, srv46522739=66, srv453567040=65, srv1013573134=0, srv751121962=80, srv1925726655=42, srv1781861026=32, srv481812578=68, srv1507447850=24, srv1395681284=18, srv522564382=71, srv875184987=92, srv1956999066=46, srv933289332=96, srv1492621600=23, srv531050543=72, srv1057320898=2, srv1652732052=28, srv1989360321=49, srv76284902=82, srv1165718431=8, srv1866558889=38, srv1462625755=22, srv241751107=56, srv927666158=95, srv1955970091=44, srv727707620=77, srv1185104802=10, srv1930739750=43, srv1179671223=9, srv44310704=64, srv2094908416=51, srv1289496794=14, srv781544735=84, srv1570673706=25, srv847711571=89, srv1216564897=12, srv172537197=30, srv823098907=87, srv294209228=58, srv1979272760=47, srv598216748=74, srv883328781=93, srv106169244=3, srv1261791637=13, srv1786642941=34, srv751764623=81, srv1092136557=4, srv87473540=91, srv1801915109=36, srv1862263035=37, srv2098775844=52, srv1783768378=33, srv1440887355=21, srv969421253=98, srv1893844518=41, srv355373880=60, srv817256657=86, srv1094039387=5, srv1429048956=20, srv381038534=63, srv351497537=59, srv1866825677=39, srv739896597=79, srv978326568=99, srv1333682714=16, srv1790039759=35, srv948546288=97, srv1956078712=45, srv1114850048=6, srv507115897=69, srv1610575862=27, srv729881032=78, srv2123983890=53, srv1403704598=19, srv1742489991=31, srv20479490=50, srv286666855=57, srv630969112=76, srv76649140=83, srv1340670106=17, srv887808902=94, srv381018484=62, srv802250739=85, srv1690592292=29, srv1610468437=26, srv1318975988=15, srv595962734=73, srv2126200699=54, srv233110714=55, srv620722305=75, srv1988596505=48} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-29 03:55:57,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-29 03:55:57,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-29 03:55:57,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-29 03:55:57,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-29 03:55:57,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-29 03:55:57,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-29 03:55:57,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-29 03:55:57,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-29 03:55:57,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-29 03:55:57,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-29 03:55:57,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-29 03:55:57,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-29 03:55:57,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-29 03:55:57,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-29 03:55:57,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-29 03:55:57,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-29 03:55:57,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-29 03:55:57,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-29 03:55:57,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-29 03:55:57,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-29 03:55:57,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-29 03:55:57,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-29 03:55:57,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-29 03:55:57,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-29 03:55:57,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-29 03:55:57,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-29 03:55:57,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-29 03:55:57,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-29 03:55:57,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-29 03:55:57,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-29 03:55:57,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-29 03:55:57,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-29 03:55:57,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-29 03:55:57,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-29 03:55:57,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-29 03:55:57,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-29 03:55:57,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-29 03:55:57,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-29 03:55:57,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-29 03:55:57,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-29 03:55:57,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-29 03:55:57,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-29 03:55:57,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-29 03:55:57,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-29 03:55:57,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-29 03:55:57,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-29 03:55:57,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-29 03:55:57,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-29 03:55:57,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-29 03:55:57,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-29 03:55:57,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-29 03:55:57,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-29 03:55:57,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-29 03:55:57,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-29 03:55:57,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-29 03:55:57,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-29 03:55:57,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-29 03:55:57,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-29 03:55:57,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-29 03:55:57,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-29 03:55:57,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-29 03:55:57,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-29 03:55:57,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-29 03:55:57,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-29 03:55:57,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-29 03:55:57,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-29 03:55:57,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-29 03:55:57,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-29 03:55:57,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-29 03:55:57,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-29 03:55:57,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-29 03:55:57,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-29 03:55:57,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-29 03:55:57,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-29 03:55:57,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-29 03:55:57,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-29 03:55:57,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-29 03:55:57,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-29 03:55:57,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-29 03:55:57,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-29 03:55:57,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-29 03:55:57,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-29 03:55:57,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-29 03:55:57,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-29 03:55:57,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-29 03:55:57,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-29 03:55:57,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-29 03:55:57,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-29 03:55:57,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-29 03:55:57,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-29 03:55:57,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-29 03:55:57,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-29 03:55:57,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-29 03:55:57,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-29 03:55:57,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-29 03:55:57,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-29 03:55:57,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-29 03:55:57,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-29 03:55:57,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-29 03:55:57,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-29 03:55:57,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-29 03:55:57,519 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04127413305097533, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05213295615870173, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04127413305097533, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.179, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24758333333333332, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-29 03:55:57,520 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-29 03:55:57,520 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18387228204394207, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04127413305097533, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05213295615870173, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04127413305097533, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.179, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24758333333333332, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-29 03:55:57,929 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 572 ms to try 5000 different iterations.  Found a solution that moves 2633 regions; Going from a computed imbalance of 0.18387228204394207 to a new imbalance of 0.01531786129638274. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02386153263608836); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03162987407155247, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29255555555555557, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02386153263608836); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012333333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04375, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-29 03:55:57,944 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1333682714:78 , srv1955970091:79 , srv1988596505:79 , srv1930739750:80 , srv355373880:80 , srv802250739:81 , srv831751529:81 , srv1395681284:82 , srv1979272760:82 , srv2123983890:82 , srv1507447850:83 , srv172537197:83 , srv1781861026:83 , srv1925726655:83 , srv20479490:83 , srv595962734:83 , srv630969112:83 , srv781544735:83 , srv887808902:83 , srv969421253:83 , srv1114850048:84 , srv1179671223:84 , srv1185104802:84 , srv1289496794:84 , srv1340670106:84 , srv1801915109:84 , srv1866825677:84 , srv46522739:84 , srv883328781:84 , srv1092136557:85 , srv1165718431:85 , srv1570673706:85 , srv1742489991:85 , srv1786642941:85 , srv1862263035:85 , srv2098775844:85 , srv241751107:85 , srv286666855:85 , srv351497537:85 , srv46717968:85 , srv522564382:85 , srv729881032:85 , srv739896597:85 , srv823098907:85 , srv847711571:85 , srv948546288:85 , srv978326568:85 , srv1013573134:86 , srv1094039387:86 , srv1690592292:86 , srv1783768378:86 , srv1873764958:86 , srv1956078712:86 , srv381038534:86 , srv44310704:86 , srv514986774:86 , srv531050543:86 , srv598216748:86 , srv727707620:86 , srv864833024:86 , srv875184987:86 , srv933289332:86 , srv102864432:87 , srv1057320898:87 , srv106169244:87 , srv1216564897:87 , srv1403704598:87 , srv1429048956:87 , srv1492621600:87 , srv1790039759:87 , srv1893844518:87 , srv1956999066:87 , srv1989360321:87 , srv2094908416:87 , srv233110714:87 , srv381018484:87 , srv817256657:87 , srv1215111640:88 , srv1440887355:88 , srv1462625755:88 , srv1866558889:88 , srv2126200699:88 , srv294209228:88 , srv37956910:88 , srv481812578:88 , srv751764623:88 , srv1148243188:89 , srv1318975988:89 , srv1610575862:89 , srv453567040:89 , srv507115897:89 , srv620722305:89 , srv751121962:89 , srv87473540:89 , srv927666158:89 , srv1261791637:90 , srv1610468437:90 , srv76284902:91 , srv76649140:92 , srv1652732052:529 }
2023-03-29 03:55:57,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-29 03:55:57,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1215111640=11, srv514986774=70, srv46717968=67, srv831751529=88, srv864833024=90, srv37956910=61, srv102864432=1, srv1148243188=7, srv1873764958=40, srv46522739=66, srv453567040=65, srv1013573134=0, srv751121962=80, srv1925726655=42, srv1781861026=32, srv481812578=68, srv1507447850=24, srv1395681284=18, srv522564382=71, srv875184987=92, srv1956999066=46, srv933289332=96, srv1492621600=23, srv531050543=72, srv1057320898=2, srv1652732052=28, srv1989360321=49, srv76284902=82, srv1165718431=8, srv1866558889=38, srv1462625755=22, srv241751107=56, srv927666158=95, srv1955970091=44, srv727707620=77, srv1185104802=10, srv1930739750=43, srv1179671223=9, srv44310704=64, srv2094908416=51, srv1289496794=14, srv781544735=84, srv1570673706=25, srv847711571=89, srv1216564897=12, srv172537197=30, srv823098907=87, srv294209228=58, srv1979272760=47, srv598216748=74, srv883328781=93, srv106169244=3, srv1261791637=13, srv1786642941=34, srv751764623=81, srv1092136557=4, srv87473540=91, srv1801915109=36, srv1862263035=37, srv2098775844=52, srv1783768378=33, srv1440887355=21, srv969421253=98, srv1893844518=41, srv355373880=60, srv817256657=86, srv1094039387=5, srv1429048956=20, srv381038534=63, srv351497537=59, srv1866825677=39, srv739896597=79, srv978326568=99, srv1333682714=16, srv1790039759=35, srv948546288=97, srv1956078712=45, srv1114850048=6, srv507115897=69, srv1610575862=27, srv729881032=78, srv2123983890=53, srv1403704598=19, srv1742489991=31, srv20479490=50, srv286666855=57, srv630969112=76, srv76649140=83, srv1340670106=17, srv887808902=94, srv381018484=62, srv802250739=85, srv1690592292=29, srv1610468437=26, srv1318975988=15, srv595962734=73, srv2126200699=54, srv233110714=55, srv620722305=75, srv1988596505=48} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-29 03:55:57,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-29 03:55:57,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-29 03:55:57,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-29 03:55:57,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-29 03:55:57,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-29 03:55:57,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-29 03:55:57,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-29 03:55:57,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-29 03:55:57,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-29 03:55:57,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-29 03:55:57,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-29 03:55:57,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-29 03:55:57,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-29 03:55:57,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-29 03:55:57,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-29 03:55:57,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-29 03:55:57,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-29 03:55:57,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-29 03:55:57,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-29 03:55:57,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-29 03:55:57,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-29 03:55:57,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-29 03:55:57,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-29 03:55:57,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-29 03:55:57,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-29 03:55:57,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-29 03:55:57,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-29 03:55:57,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-29 03:55:57,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-29 03:55:57,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-29 03:55:57,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-29 03:55:57,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-29 03:55:57,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-29 03:55:57,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-29 03:55:57,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-29 03:55:57,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-29 03:55:57,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-29 03:55:57,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-29 03:55:57,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-29 03:55:57,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-29 03:55:57,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-29 03:55:57,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-29 03:55:57,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-29 03:55:57,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-29 03:55:57,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-29 03:55:57,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-29 03:55:57,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-29 03:55:57,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-29 03:55:57,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-29 03:55:57,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-29 03:55:57,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-29 03:55:57,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-29 03:55:57,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-29 03:55:57,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-29 03:55:57,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-29 03:55:57,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-29 03:55:57,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-29 03:55:57,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-29 03:55:57,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-29 03:55:57,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-29 03:55:57,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-29 03:55:57,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-29 03:55:57,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-29 03:55:57,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-29 03:55:57,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-29 03:55:57,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-29 03:55:57,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-29 03:55:57,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-29 03:55:57,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-29 03:55:57,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-29 03:55:57,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-29 03:55:57,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-29 03:55:57,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-29 03:55:57,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-29 03:55:57,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-29 03:55:57,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-29 03:55:57,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-29 03:55:57,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-29 03:55:57,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-29 03:55:57,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-29 03:55:57,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-29 03:55:57,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-29 03:55:57,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-29 03:55:57,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-29 03:55:57,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-29 03:55:57,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-29 03:55:57,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-29 03:55:57,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-29 03:55:57,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-29 03:55:57,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-29 03:55:57,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-29 03:55:57,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-29 03:55:57,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-29 03:55:57,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-29 03:55:57,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-29 03:55:57,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-29 03:55:57,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-29 03:55:57,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-29 03:55:57,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-29 03:55:57,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-29 03:55:57,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-29 03:55:58,135 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02386153263608836); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03162987407155247, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02386153263608836); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012333333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04375, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-29 03:55:58,135 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-29 03:55:58,136 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.015299424636328102, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02386153263608836); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03162987407155247, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02386153263608836); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012333333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04375, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-29 03:55:58,496 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 501 ms to try 5000 different iterations.  Found a solution that moves 939 regions; Going from a computed imbalance of 0.015299424636328102 to a new imbalance of 6.186360750125433E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007339758434175736); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00478493708376849); MoveCostFunction : (multiplier=7.0, imbalance=0.10433333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007339758434175736); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0045); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-29 03:55:58,499 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1786642941:82 , srv1866558889:82 , srv507115897:82 , srv927666158:82 , srv933289332:82 , srv106169244:83 , srv1261791637:83 , srv1318975988:83 , srv1429048956:83 , srv1507447850:83 , srv1790039759:83 , srv233110714:83 , srv37956910:83 , srv76284902:83 , srv76649140:83 , srv1148243188:84 , srv1333682714:84 , srv1610468437:84 , srv1610575862:84 , srv1801915109:84 , srv1873764958:84 , srv1956078712:84 , srv2098775844:84 , srv381038534:84 , srv531050543:84 , srv87473540:84 , srv1013573134:85 , srv1094039387:85 , srv1185104802:85 , srv1215111640:85 , srv1403704598:85 , srv1690592292:85 , srv172537197:85 , srv351497537:85 , srv44310704:85 , srv481812578:85 , srv514986774:85 , srv522564382:85 , srv729881032:85 , srv739896597:85 , srv751121962:85 , srv751764623:85 , srv781544735:85 , srv831751529:85 , srv875184987:85 , srv948546288:85 , srv969421253:85 , srv1057320898:86 , srv1165718431:86 , srv1179671223:86 , srv1395681284:86 , srv1440887355:86 , srv1492621600:86 , srv1570673706:86 , srv1742489991:86 , srv1862263035:86 , srv1955970091:86 , srv1979272760:86 , srv20479490:86 , srv2094908416:86 , srv2123983890:86 , srv241751107:86 , srv294209228:86 , srv381018484:86 , srv453567040:86 , srv630969112:86 , srv727707620:86 , srv817256657:86 , srv864833024:86 , srv883328781:86 , srv102864432:87 , srv1092136557:87 , srv1114850048:87 , srv1216564897:87 , srv1340670106:87 , srv1462625755:87 , srv1781861026:87 , srv1893844518:87 , srv1956999066:87 , srv1988596505:87 , srv2126200699:87 , srv355373880:87 , srv46522739:87 , srv595962734:87 , srv620722305:87 , srv823098907:87 , srv847711571:87 , srv978326568:87 , srv1289496794:88 , srv1783768378:88 , srv1866825677:88 , srv1925726655:88 , srv1930739750:88 , srv1989360321:88 , srv286666855:88 , srv46717968:88 , srv598216748:88 , srv887808902:88 , srv802250739:90 , srv1652732052:537 }
2023-03-29 03:55:58,505 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-29 03:55:58,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1215111640=11, srv514986774=70, srv46717968=67, srv831751529=88, srv864833024=90, srv37956910=61, srv102864432=1, srv1148243188=7, srv1873764958=40, srv46522739=66, srv453567040=65, srv1013573134=0, srv751121962=80, srv1925726655=42, srv1781861026=32, srv481812578=68, srv1507447850=24, srv1395681284=18, srv522564382=71, srv875184987=92, srv1956999066=46, srv933289332=96, srv1492621600=23, srv531050543=72, srv1057320898=2, srv1652732052=28, srv1989360321=49, srv76284902=82, srv1165718431=8, srv1866558889=38, srv1462625755=22, srv241751107=56, srv927666158=95, srv1955970091=44, srv727707620=77, srv1185104802=10, srv1930739750=43, srv1179671223=9, srv44310704=64, srv2094908416=51, srv1289496794=14, srv781544735=84, srv1570673706=25, srv847711571=89, srv1216564897=12, srv172537197=30, srv823098907=87, srv294209228=58, srv1979272760=47, srv598216748=74, srv883328781=93, srv106169244=3, srv1261791637=13, srv1786642941=34, srv751764623=81, srv1092136557=4, srv87473540=91, srv1801915109=36, srv1862263035=37, srv2098775844=52, srv1783768378=33, srv1440887355=21, srv969421253=98, srv1893844518=41, srv355373880=60, srv817256657=86, srv1094039387=5, srv1429048956=20, srv381038534=63, srv351497537=59, srv1866825677=39, srv739896597=79, srv978326568=99, srv1333682714=16, srv1790039759=35, srv948546288=97, srv1956078712=45, srv1114850048=6, srv507115897=69, srv1610575862=27, srv729881032=78, srv2123983890=53, srv1403704598=19, srv1742489991=31, srv20479490=50, srv286666855=57, srv630969112=76, srv76649140=83, srv1340670106=17, srv887808902=94, srv381018484=62, srv802250739=85, srv1690592292=29, srv1610468437=26, srv1318975988=15, srv595962734=73, srv2126200699=54, srv233110714=55, srv620722305=75, srv1988596505=48} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-29 03:55:58,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-29 03:55:58,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-29 03:55:58,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-29 03:55:58,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-29 03:55:58,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-29 03:55:58,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-29 03:55:58,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-29 03:55:58,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-29 03:55:58,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-29 03:55:58,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-29 03:55:58,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-29 03:55:58,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-29 03:55:58,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-29 03:55:58,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-29 03:55:58,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-29 03:55:58,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-29 03:55:58,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-29 03:55:58,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-29 03:55:58,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-29 03:55:58,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-29 03:55:58,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-29 03:55:58,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-29 03:55:58,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-29 03:55:58,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-29 03:55:58,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-29 03:55:58,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-29 03:55:58,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-29 03:55:58,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-29 03:55:58,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-29 03:55:58,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-29 03:55:58,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-29 03:55:58,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-29 03:55:58,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-29 03:55:58,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-29 03:55:58,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-29 03:55:58,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-29 03:55:58,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-29 03:55:58,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-29 03:55:58,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-29 03:55:58,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-29 03:55:58,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-29 03:55:58,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-29 03:55:58,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-29 03:55:58,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-29 03:55:58,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-29 03:55:58,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-29 03:55:58,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-29 03:55:58,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-29 03:55:58,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-29 03:55:58,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-29 03:55:58,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-29 03:55:58,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-29 03:55:58,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-29 03:55:58,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-29 03:55:58,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-29 03:55:58,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-29 03:55:58,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-29 03:55:58,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-29 03:55:58,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-29 03:55:58,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-29 03:55:58,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-29 03:55:58,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-29 03:55:58,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-29 03:55:58,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-29 03:55:58,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-29 03:55:58,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-29 03:55:58,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-29 03:55:58,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-29 03:55:58,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-29 03:55:58,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-29 03:55:58,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-29 03:55:58,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-29 03:55:58,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-29 03:55:58,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-29 03:55:58,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-29 03:55:58,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-29 03:55:58,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-29 03:55:58,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-29 03:55:58,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-29 03:55:58,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-29 03:55:58,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-29 03:55:58,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-29 03:55:58,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-29 03:55:58,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-29 03:55:58,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-29 03:55:58,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-29 03:55:58,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-29 03:55:58,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-29 03:55:58,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-29 03:55:58,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-29 03:55:58,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-29 03:55:58,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-29 03:55:58,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-29 03:55:58,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-29 03:55:58,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-29 03:55:58,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-29 03:55:58,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-29 03:55:58,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-29 03:55:58,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-29 03:55:58,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-29 03:55:58,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-29 03:55:58,699 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007339758434175736); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00478493708376849); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007339758434175736); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0045); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-29 03:55:58,699 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-29 03:55:58,704 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=6.120610564818544E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007339758434175736); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00478493708376849); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007339758434175736); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0045); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-29 03:55:59,104 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 544 ms to try 5000 different iterations.  Found a solution that moves 233 regions; Going from a computed imbalance of 6.120610564818544E-4 to a new imbalance of 1.951887882409255E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017727200782610348); MoveCostFunction : (multiplier=7.0, imbalance=0.02588888888888889, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-29 03:55:59,106 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1013573134:83 , srv1185104802:83 , srv1216564897:83 , srv1507447850:83 , srv1988596505:83 , srv381038534:83 , srv453567040:83 , srv46717968:83 , srv102864432:84 , srv1057320898:84 , srv1289496794:84 , srv1333682714:84 , srv1340670106:84 , srv1429048956:84 , srv1462625755:84 , srv1610575862:84 , srv1690592292:84 , srv1783768378:84 , srv1786642941:84 , srv1866825677:84 , srv1893844518:84 , srv1956999066:84 , srv1989360321:84 , srv2126200699:84 , srv286666855:84 , srv294209228:84 , srv381018484:84 , srv44310704:84 , srv507115897:84 , srv514986774:84 , srv595962734:84 , srv76284902:84 , srv76649140:84 , srv802250739:84 , srv887808902:84 , srv978326568:84 , srv106169244:85 , srv1092136557:85 , srv1094039387:85 , srv1148243188:85 , srv1165718431:85 , srv1215111640:85 , srv1261791637:85 , srv1318975988:85 , srv1395681284:85 , srv1403704598:85 , srv1440887355:85 , srv1492621600:85 , srv1570673706:85 , srv1610468437:85 , srv172537197:85 , srv1742489991:85 , srv1781861026:85 , srv1790039759:85 , srv1801915109:85 , srv1862263035:85 , srv1866558889:85 , srv1873764958:85 , srv1925726655:85 , srv1930739750:85 , srv1955970091:85 , srv1956078712:85 , srv20479490:85 , srv2094908416:85 , srv2098775844:85 , srv2123983890:85 , srv241751107:85 , srv351497537:85 , srv355373880:85 , srv37956910:85 , srv46522739:85 , srv481812578:85 , srv531050543:85 , srv598216748:85 , srv620722305:85 , srv630969112:85 , srv727707620:85 , srv729881032:85 , srv739896597:85 , srv751121962:85 , srv781544735:85 , srv817256657:85 , srv823098907:85 , srv831751529:85 , srv847711571:85 , srv864833024:85 , srv87473540:85 , srv875184987:85 , srv883328781:85 , srv927666158:85 , srv948546288:85 , srv1114850048:86 , srv1179671223:86 , srv1979272760:86 , srv233110714:86 , srv751764623:86 , srv933289332:86 , srv969421253:86 , srv522564382:87 , srv1652732052:620 }
2023-03-29 03:55:59,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-29 03:55:59,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1215111640=11, srv514986774=70, srv46717968=67, srv831751529=88, srv864833024=90, srv37956910=61, srv102864432=1, srv1148243188=7, srv1873764958=40, srv46522739=66, srv453567040=65, srv1013573134=0, srv751121962=80, srv1925726655=42, srv1781861026=32, srv481812578=68, srv1507447850=24, srv1395681284=18, srv522564382=71, srv875184987=92, srv1956999066=46, srv933289332=96, srv1492621600=23, srv531050543=72, srv1057320898=2, srv1652732052=28, srv1989360321=49, srv76284902=82, srv1165718431=8, srv1866558889=38, srv1462625755=22, srv241751107=56, srv927666158=95, srv1955970091=44, srv727707620=77, srv1185104802=10, srv1930739750=43, srv1179671223=9, srv44310704=64, srv2094908416=51, srv1289496794=14, srv781544735=84, srv1570673706=25, srv847711571=89, srv1216564897=12, srv172537197=30, srv823098907=87, srv294209228=58, srv1979272760=47, srv598216748=74, srv883328781=93, srv106169244=3, srv1261791637=13, srv1786642941=34, srv751764623=81, srv1092136557=4, srv87473540=91, srv1801915109=36, srv1862263035=37, srv2098775844=52, srv1783768378=33, srv1440887355=21, srv969421253=98, srv1893844518=41, srv355373880=60, srv817256657=86, srv1094039387=5, srv1429048956=20, srv381038534=63, srv351497537=59, srv1866825677=39, srv739896597=79, srv978326568=99, srv1333682714=16, srv1790039759=35, srv948546288=97, srv1956078712=45, srv1114850048=6, srv507115897=69, srv1610575862=27, srv729881032=78, srv2123983890=53, srv1403704598=19, srv1742489991=31, srv20479490=50, srv286666855=57, srv630969112=76, srv76649140=83, srv1340670106=17, srv887808902=94, srv381018484=62, srv802250739=85, srv1690592292=29, srv1610468437=26, srv1318975988=15, srv595962734=73, srv2126200699=54, srv233110714=55, srv620722305=75, srv1988596505=48} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-29 03:55:59,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-29 03:55:59,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-29 03:55:59,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-29 03:55:59,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-29 03:55:59,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-29 03:55:59,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-29 03:55:59,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-29 03:55:59,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-29 03:55:59,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-29 03:55:59,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-29 03:55:59,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-29 03:55:59,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-29 03:55:59,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-29 03:55:59,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-29 03:55:59,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-29 03:55:59,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-29 03:55:59,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-29 03:55:59,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-29 03:55:59,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-29 03:55:59,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-29 03:55:59,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-29 03:55:59,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-29 03:55:59,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-29 03:55:59,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-29 03:55:59,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-29 03:55:59,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-29 03:55:59,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-29 03:55:59,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-29 03:55:59,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-29 03:55:59,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-29 03:55:59,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-29 03:55:59,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-29 03:55:59,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-29 03:55:59,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-29 03:55:59,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-29 03:55:59,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-29 03:55:59,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-29 03:55:59,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-29 03:55:59,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-29 03:55:59,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-29 03:55:59,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-29 03:55:59,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-29 03:55:59,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-29 03:55:59,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-29 03:55:59,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-29 03:55:59,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-29 03:55:59,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-29 03:55:59,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-29 03:55:59,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-29 03:55:59,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-29 03:55:59,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-29 03:55:59,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-29 03:55:59,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-29 03:55:59,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-29 03:55:59,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-29 03:55:59,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-29 03:55:59,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-29 03:55:59,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-29 03:55:59,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-29 03:55:59,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-29 03:55:59,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-29 03:55:59,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-29 03:55:59,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-29 03:55:59,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-29 03:55:59,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-29 03:55:59,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-29 03:55:59,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-29 03:55:59,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-29 03:55:59,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-29 03:55:59,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-29 03:55:59,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-29 03:55:59,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-29 03:55:59,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-29 03:55:59,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-29 03:55:59,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-29 03:55:59,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-29 03:55:59,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-29 03:55:59,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-29 03:55:59,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-29 03:55:59,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-29 03:55:59,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-29 03:55:59,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-29 03:55:59,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-29 03:55:59,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-29 03:55:59,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-29 03:55:59,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-29 03:55:59,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-29 03:55:59,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-29 03:55:59,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-29 03:55:59,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-29 03:55:59,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-29 03:55:59,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-29 03:55:59,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-29 03:55:59,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-29 03:55:59,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-29 03:55:59,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-29 03:55:59,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-29 03:55:59,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-29 03:55:59,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-29 03:55:59,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-29 03:55:59,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-29 03:55:59,303 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.7887377953325223E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017727200782610348); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-29 03:55:59,303 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1013573134:83 , srv1185104802:83 , srv1216564897:83 , srv1507447850:83 , srv1988596505:83 , srv381038534:83 , srv453567040:83 , srv46717968:83 , srv102864432:84 , srv1057320898:84 , srv1289496794:84 , srv1333682714:84 , srv1340670106:84 , srv1429048956:84 , srv1462625755:84 , srv1610575862:84 , srv1690592292:84 , srv1783768378:84 , srv1786642941:84 , srv1866825677:84 , srv1893844518:84 , srv1956999066:84 , srv1989360321:84 , srv2126200699:84 , srv286666855:84 , srv294209228:84 , srv381018484:84 , srv44310704:84 , srv507115897:84 , srv514986774:84 , srv595962734:84 , srv76284902:84 , srv76649140:84 , srv802250739:84 , srv887808902:84 , srv978326568:84 , srv106169244:85 , srv1092136557:85 , srv1094039387:85 , srv1148243188:85 , srv1165718431:85 , srv1215111640:85 , srv1261791637:85 , srv1318975988:85 , srv1395681284:85 , srv1403704598:85 , srv1440887355:85 , srv1492621600:85 , srv1570673706:85 , srv1610468437:85 , srv172537197:85 , srv1742489991:85 , srv1781861026:85 , srv1790039759:85 , srv1801915109:85 , srv1862263035:85 , srv1866558889:85 , srv1873764958:85 , srv1925726655:85 , srv1930739750:85 , srv1955970091:85 , srv1956078712:85 , srv20479490:85 , srv2094908416:85 , srv2098775844:85 , srv2123983890:85 , srv241751107:85 , srv351497537:85 , srv355373880:85 , srv37956910:85 , srv46522739:85 , srv481812578:85 , srv531050543:85 , srv598216748:85 , srv620722305:85 , srv630969112:85 , srv727707620:85 , srv729881032:85 , srv739896597:85 , srv751121962:85 , srv781544735:85 , srv817256657:85 , srv823098907:85 , srv831751529:85 , srv847711571:85 , srv864833024:85 , srv87473540:85 , srv875184987:85 , srv883328781:85 , srv927666158:85 , srv948546288:85 , srv1114850048:86 , srv1179671223:86 , srv1979272760:86 , srv233110714:86 , srv751764623:86 , srv933289332:86 , srv969421253:86 , srv522564382:87 , srv1652732052:620 }
2023-03-29 03:55:59,347 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=432 (was 409) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 169), AvailableMemoryMB=7050 (was 7532)
2023-03-29 03:55:59,366 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=262, MaxFileDescriptor=60000, SystemLoadAverage=432, ProcessCount=169, AvailableMemoryMB=7050
2023-03-29 03:55:59,366 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-29 03:55:59,367 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-03-29 03:55:59,381 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1073231916:84 , srv1238358657:84 , srv1240057399:84 , srv1304848414:84 , srv1439162023:84 , srv1472952765:84 , srv1502648604:84 , srv161149958:84 , srv1709603028:84 , srv1727885843:84 , srv1764866725:84 , srv1791279997:84 , srv190124992:84 , srv1926940418:84 , srv196798192:84 , srv2012468811:84 , srv2023228779:84 , srv2039852849:84 , srv2145269324:84 , srv378732318:84 , srv42122152:84 , srv509285086:84 , srv563317429:84 , srv607938353:84 , srv710926732:84 , srv739482247:84 , srv81212284:84 , srv912932260:84 , srv928700297:84 , srv622204629:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-03-29 03:55:59,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-29 03:55:59,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1439162023=4, srv81212284=27, srv1709603028=8, srv2023228779=16, srv161149958=7, srv563317429=22, srv196798192=14, srv622204629=24, srv928700297=29, srv607938353=23, srv1764866725=10, srv42122152=20, srv1727885843=9, srv2012468811=15, srv1926940418=13, srv2039852849=17, srv378732318=19, srv912932260=28, srv1472952765=5, srv509285086=21, srv1238358657=1, srv190124992=12, srv1304848414=3, srv710926732=25, srv1791279997=11, srv1502648604=6, srv1240057399=2, srv2145269324=18, srv739482247=26, srv1073231916=0} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-29 03:55:59,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-29 03:55:59,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-29 03:55:59,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-29 03:55:59,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-29 03:55:59,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-29 03:55:59,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-29 03:55:59,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-29 03:55:59,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-29 03:55:59,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-29 03:55:59,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-29 03:55:59,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-29 03:55:59,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-29 03:55:59,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-29 03:55:59,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-29 03:55:59,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-29 03:55:59,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-29 03:55:59,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-29 03:55:59,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-29 03:55:59,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-29 03:55:59,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-29 03:55:59,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-29 03:55:59,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-29 03:55:59,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-29 03:55:59,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-29 03:55:59,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-29 03:55:59,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-29 03:55:59,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-29 03:55:59,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-29 03:55:59,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-29 03:55:59,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-29 03:55:59,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-03-29 03:55:59,422 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0726280365975733, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-29 03:55:59,422 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912421891836076, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0726280365975733, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-03-29 03:57:59,404 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5953407 different iterations.  Found a solution that moves 1860 regions; Going from a computed imbalance of 0.9912421891836076 to a new imbalance of 4.341332789166274E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6888888888888889, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-29 03:57:59,411 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1073231916:90 , srv1238358657:90 , srv1240057399:90 , srv1304848414:90 , srv1439162023:90 , srv1472952765:90 , srv1502648604:90 , srv161149958:90 , srv1709603028:90 , srv1727885843:90 , srv1764866725:90 , srv1791279997:90 , srv190124992:90 , srv1926940418:90 , srv196798192:90 , srv2012468811:90 , srv2023228779:90 , srv2039852849:90 , srv2145269324:90 , srv378732318:90 , srv42122152:90 , srv509285086:90 , srv563317429:90 , srv607938353:90 , srv622204629:90 , srv710926732:90 , srv739482247:90 , srv81212284:90 , srv912932260:90 , srv928700297:90 }
2023-03-29 03:57:59,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-29 03:57:59,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1439162023=4, srv81212284=27, srv1709603028=8, srv2023228779=16, srv161149958=7, srv563317429=22, srv196798192=14, srv622204629=24, srv928700297=29, srv607938353=23, srv1764866725=10, srv42122152=20, srv1727885843=9, srv2012468811=15, srv1926940418=13, srv2039852849=17, srv378732318=19, srv912932260=28, srv1472952765=5, srv509285086=21, srv1238358657=1, srv190124992=12, srv1304848414=3, srv710926732=25, srv1791279997=11, srv1502648604=6, srv1240057399=2, srv2145269324=18, srv739482247=26, srv1073231916=0} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-29 03:57:59,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-29 03:57:59,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-29 03:57:59,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-29 03:57:59,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-29 03:57:59,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-29 03:57:59,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-29 03:57:59,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-29 03:57:59,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-29 03:57:59,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-29 03:57:59,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-29 03:57:59,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-29 03:57:59,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-29 03:57:59,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-29 03:57:59,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-29 03:57:59,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-29 03:57:59,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-29 03:57:59,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-29 03:57:59,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-29 03:57:59,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-29 03:57:59,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-29 03:57:59,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-29 03:57:59,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-29 03:57:59,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-29 03:57:59,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-29 03:57:59,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-29 03:57:59,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-29 03:57:59,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-29 03:57:59,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-29 03:57:59,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-29 03:57:59,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-29 03:57:59,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-03-29 03:57:59,429 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-29 03:57:59,429 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1073231916:90 , srv1238358657:90 , srv1240057399:90 , srv1304848414:90 , srv1439162023:90 , srv1472952765:90 , srv1502648604:90 , srv161149958:90 , srv1709603028:90 , srv1727885843:90 , srv1764866725:90 , srv1791279997:90 , srv190124992:90 , srv1926940418:90 , srv196798192:90 , srv2012468811:90 , srv2023228779:90 , srv2039852849:90 , srv2145269324:90 , srv378732318:90 , srv42122152:90 , srv509285086:90 , srv563317429:90 , srv607938353:90 , srv622204629:90 , srv710926732:90 , srv739482247:90 , srv81212284:90 , srv912932260:90 , srv928700297:90 }
2023-03-29 03:57:59,444 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 262), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=244 (was 432), ProcessCount=171 (was 169) - ProcessCount LEAK? -, AvailableMemoryMB=7358 (was 7050) - AvailableMemoryMB LEAK? -