2023-04-03 10:00:52,495 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-04-03 10:00:52,849 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-04-03 10:00:52,952 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-04-03 10:00:53,006 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-03 10:00:53,012 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-04-03 10:00:53,023 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=319, ProcessCount=166, AvailableMemoryMB=4832
2023-04-03 10:00:53,025 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-03 10:00:53,025 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-04-03 10:00:53,159 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1003334649:84 , srv1029292851:84 , srv1032605757:84 , srv1035555763:84 , srv1056075179:84 , srv1110416497:84 , srv111864724:84 , srv1141406499:84 , srv1161486954:84 , srv1180459024:84 , srv1185396553:84 , srv1186985074:84 , srv119080410:84 , srv119871269:84 , srv1201994079:84 , srv1273937045:84 , srv127452002:84 , srv1281343110:84 , srv128851781:84 , srv1291581568:84 , srv1292040204:84 , srv1324114884:84 , srv1328868920:84 , srv132893318:84 , srv1348240354:84 , srv1353967962:84 , srv139167575:84 , srv1429150942:84 , srv1532399695:84 , srv1551005160:84 , srv1554763571:84 , srv1561948300:84 , srv1568881768:84 , srv1569726993:84 , srv1582988894:84 , srv1595574430:84 , srv1623230812:84 , srv1625111152:84 , srv163064835:84 , srv1644305959:84 , srv1684098858:84 , srv1740411852:84 , srv1743190360:84 , srv1760507329:84 , srv177223491:84 , srv1794965544:84 , srv187185342:84 , srv190183033:84 , srv1928447129:84 , srv19445090:84 , srv1946383861:84 , srv1968697751:84 , srv1992490954:84 , srv1992999429:84 , srv1997051619:84 , srv2036499128:84 , srv2064505362:84 , srv2082287730:84 , srv2083015882:84 , srv2094322246:84 , srv2104135761:84 , srv2119896556:84 , srv2131111515:84 , srv2138654190:84 , srv215258627:84 , srv269874159:84 , srv302658654:84 , srv318056279:84 , srv330124450:84 , srv34277612:84 , srv344887986:84 , srv345717143:84 , srv404413399:84 , srv433663735:84 , srv451743634:84 , srv472344242:84 , srv515417680:84 , srv519697047:84 , srv52189100:84 , srv546749091:84 , srv54956873:84 , srv589902059:84 , srv605086189:84 , srv643854097:84 , srv686494736:84 , srv716520265:84 , srv718788612:84 , srv742183441:84 , srv761822490:84 , srv788817284:84 , srv812261468:84 , srv816639848:84 , srv824580383:84 , srv92348499:84 , srv941975545:84 , srv94210750:84 , srv947610687:84 , srv969265360:84 , srv989590250:84 , srv1542729188:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-04-03 10:00:53,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-03 10:00:53,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv187185342=47, srv812261468=91, srv1032605757=2, srv1946383861=51, srv2036499128=56, srv1595574430=36, srv1328868920=22, srv969265360=98, srv1644305959=40, srv546749091=80, srv1760507329=44, srv2104135761=61, srv1185396553=10, srv1992999429=54, srv1568881768=33, srv54956873=81, srv1623230812=37, srv2082287730=58, srv1141406499=7, srv1554763571=31, srv215258627=65, srv1056075179=4, srv2119896556=62, srv1569726993=34, srv761822490=89, srv1740411852=42, srv1003334649=0, srv1180459024=9, srv1273937045=15, srv433663735=74, srv1743190360=43, srv111864724=6, srv92348499=94, srv119080410=12, srv1532399695=28, srv643854097=84, srv686494736=85, srv1281343110=17, srv1348240354=24, srv404413399=73, srv132893318=23, srv589902059=82, srv128851781=18, srv1201994079=14, srv605086189=83, srv302658654=67, srv1324114884=21, srv139167575=26, srv1684098858=41, srv1968697751=52, srv94210750=96, srv34277612=70, srv1794965544=46, srv1029292851=1, srv816639848=92, srv824580383=93, srv1992490954=53, srv19445090=50, srv2131111515=63, srv2138654190=64, srv344887986=71, srv788817284=90, srv515417680=77, srv318056279=68, srv2094322246=60, srv190183033=48, srv127452002=16, srv2064505362=57, srv472344242=76, srv269874159=66, srv718788612=87, srv1551005160=30, srv1997051619=55, srv345717143=72, srv119871269=13, srv1582988894=35, srv716520265=86, srv1542729188=29, srv1561948300=32, srv519697047=78, srv52189100=79, srv989590250=99, srv1625111152=38, srv742183441=88, srv1353967962=25, srv1110416497=5, srv1186985074=11, srv1291581568=19, srv177223491=45, srv330124450=69, srv163064835=39, srv2083015882=59, srv941975545=95, srv1928447129=49, srv1035555763=3, srv1429150942=27, srv947610687=97, srv1161486954=8, srv451743634=75, srv1292040204=20} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-03 10:00:53,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-03 10:00:53,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-03 10:00:53,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-03 10:00:53,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-03 10:00:53,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-03 10:00:53,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-03 10:00:53,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-03 10:00:53,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-03 10:00:53,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-03 10:00:53,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-03 10:00:53,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-03 10:00:53,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-03 10:00:53,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-03 10:00:53,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-03 10:00:53,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-03 10:00:53,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-03 10:00:53,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-03 10:00:53,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-03 10:00:53,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-03 10:00:53,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-03 10:00:53,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-03 10:00:53,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-03 10:00:53,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-03 10:00:53,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-03 10:00:53,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-03 10:00:53,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-03 10:00:53,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-03 10:00:53,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-03 10:00:53,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-03 10:00:53,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-03 10:00:53,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-03 10:00:53,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-03 10:00:53,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-03 10:00:53,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-03 10:00:53,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-03 10:00:53,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-03 10:00:53,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-03 10:00:53,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-03 10:00:53,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-03 10:00:53,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-03 10:00:53,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-03 10:00:53,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-03 10:00:53,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-03 10:00:53,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-03 10:00:53,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-03 10:00:53,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-03 10:00:53,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-03 10:00:53,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-03 10:00:53,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-03 10:00:53,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-03 10:00:53,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-03 10:00:53,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-03 10:00:53,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-03 10:00:53,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-03 10:00:53,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-03 10:00:53,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-03 10:00:53,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-03 10:00:53,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-03 10:00:53,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-03 10:00:53,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-03 10:00:53,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-03 10:00:53,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-03 10:00:53,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-03 10:00:53,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-03 10:00:53,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-03 10:00:53,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-03 10:00:53,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-03 10:00:53,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-03 10:00:53,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-03 10:00:53,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-03 10:00:53,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-03 10:00:53,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-03 10:00:53,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-03 10:00:53,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-03 10:00:53,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-03 10:00:53,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-03 10:00:53,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-03 10:00:53,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-03 10:00:53,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-03 10:00:53,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-03 10:00:53,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-03 10:00:53,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-03 10:00:53,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-03 10:00:53,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-03 10:00:53,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-03 10:00:53,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-03 10:00:53,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-03 10:00:53,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-03 10:00:53,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-03 10:00:53,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-03 10:00:53,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-03 10:00:53,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-03 10:00:53,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-03 10:00:53,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-03 10:00:53,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-03 10:00:53,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-03 10:00:53,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-03 10:00:53,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-03 10:00:53,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-03 10:00:53,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-03 10:00:53,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-03 10:00:53,568 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-03 10:00:53,568 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-03 10:00:53,569 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-03 10:00:54,120 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 825 ms to try 5000 different iterations.  Found a solution that moves 4690 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1818089360140734. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03793326423113138, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05336488965426095, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5211111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03793326423113138, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17683333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24608333333333332, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-03 10:00:54,135 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv404413399:82 , srv969265360:82 , srv1056075179:83 , srv132893318:83 , srv1568881768:83 , srv1569726993:83 , srv19445090:83 , srv2036499128:83 , srv2094322246:83 , srv344887986:83 , srv515417680:83 , srv92348499:83 , srv1141406499:84 , srv1281343110:84 , srv1595574430:84 , srv1760507329:84 , srv1992490954:84 , srv2131111515:84 , srv34277612:84 , srv345717143:84 , srv589902059:84 , srv947610687:84 , srv1035555763:85 , srv1201994079:85 , srv1273937045:85 , srv1348240354:85 , srv139167575:85 , srv1644305959:85 , srv1684098858:85 , srv1743190360:85 , srv187185342:85 , srv2119896556:85 , srv318056279:85 , srv716520265:85 , srv788817284:85 , srv824580383:85 , srv989590250:85 , srv128851781:86 , srv1291581568:86 , srv1292040204:86 , srv1328868920:86 , srv1353967962:86 , srv1429150942:86 , srv1551005160:86 , srv1997051619:86 , srv2082287730:86 , srv2138654190:86 , srv742183441:86 , srv816639848:86 , srv94210750:86 , srv1003334649:87 , srv1582988894:87 , srv1623230812:87 , srv163064835:87 , srv177223491:87 , srv1968697751:87 , srv1992999429:87 , srv215258627:87 , srv451743634:87 , srv472344242:87 , srv52189100:87 , srv605086189:87 , srv812261468:87 , srv1029292851:88 , srv111864724:88 , srv1161486954:88 , srv1185396553:88 , srv119080410:88 , srv119871269:88 , srv1532399695:88 , srv1740411852:88 , srv1928447129:88 , srv1946383861:88 , srv2064505362:88 , srv2083015882:88 , srv2104135761:88 , srv330124450:88 , srv519697047:88 , srv718788612:88 , srv941975545:88 , srv1032605757:89 , srv1110416497:89 , srv1554763571:89 , srv1561948300:89 , srv1625111152:89 , srv269874159:89 , srv546749091:89 , srv643854097:89 , srv1186985074:90 , srv127452002:90 , srv1794965544:90 , srv302658654:90 , srv54956873:90 , srv1324114884:91 , srv761822490:91 , srv190183033:92 , srv433663735:93 , srv1180459024:95 , srv686494736:95 , srv1542729188:427 }
2023-04-03 10:00:54,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-03 10:00:54,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv187185342=47, srv812261468=91, srv1032605757=2, srv1946383861=51, srv2036499128=56, srv1595574430=36, srv1328868920=22, srv969265360=98, srv1644305959=40, srv546749091=80, srv1760507329=44, srv2104135761=61, srv1185396553=10, srv1992999429=54, srv1568881768=33, srv54956873=81, srv1623230812=37, srv2082287730=58, srv1141406499=7, srv1554763571=31, srv215258627=65, srv1056075179=4, srv2119896556=62, srv1569726993=34, srv761822490=89, srv1740411852=42, srv1003334649=0, srv1180459024=9, srv1273937045=15, srv433663735=74, srv1743190360=43, srv111864724=6, srv92348499=94, srv119080410=12, srv1532399695=28, srv643854097=84, srv686494736=85, srv1281343110=17, srv1348240354=24, srv404413399=73, srv132893318=23, srv589902059=82, srv128851781=18, srv1201994079=14, srv605086189=83, srv302658654=67, srv1324114884=21, srv139167575=26, srv1684098858=41, srv1968697751=52, srv94210750=96, srv34277612=70, srv1794965544=46, srv1029292851=1, srv816639848=92, srv824580383=93, srv1992490954=53, srv19445090=50, srv2131111515=63, srv2138654190=64, srv344887986=71, srv788817284=90, srv515417680=77, srv318056279=68, srv2094322246=60, srv190183033=48, srv127452002=16, srv2064505362=57, srv472344242=76, srv269874159=66, srv718788612=87, srv1551005160=30, srv1997051619=55, srv345717143=72, srv119871269=13, srv1582988894=35, srv716520265=86, srv1542729188=29, srv1561948300=32, srv519697047=78, srv52189100=79, srv989590250=99, srv1625111152=38, srv742183441=88, srv1353967962=25, srv1110416497=5, srv1186985074=11, srv1291581568=19, srv177223491=45, srv330124450=69, srv163064835=39, srv2083015882=59, srv941975545=95, srv1928447129=49, srv1035555763=3, srv1429150942=27, srv947610687=97, srv1161486954=8, srv451743634=75, srv1292040204=20} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-03 10:00:54,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-03 10:00:54,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-03 10:00:54,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-03 10:00:54,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-03 10:00:54,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-03 10:00:54,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-03 10:00:54,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-03 10:00:54,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-03 10:00:54,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-03 10:00:54,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-03 10:00:54,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-03 10:00:54,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-03 10:00:54,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-03 10:00:54,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-03 10:00:54,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-03 10:00:54,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-03 10:00:54,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-03 10:00:54,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-03 10:00:54,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-03 10:00:54,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-03 10:00:54,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-03 10:00:54,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-03 10:00:54,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-03 10:00:54,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-03 10:00:54,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-03 10:00:54,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-03 10:00:54,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-03 10:00:54,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-03 10:00:54,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-03 10:00:54,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-03 10:00:54,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-03 10:00:54,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-03 10:00:54,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-03 10:00:54,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-03 10:00:54,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-03 10:00:54,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-03 10:00:54,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-03 10:00:54,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-03 10:00:54,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-03 10:00:54,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-03 10:00:54,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-03 10:00:54,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-03 10:00:54,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-03 10:00:54,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-03 10:00:54,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-03 10:00:54,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-03 10:00:54,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-03 10:00:54,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-03 10:00:54,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-03 10:00:54,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-03 10:00:54,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-03 10:00:54,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-03 10:00:54,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-03 10:00:54,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-03 10:00:54,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-03 10:00:54,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-03 10:00:54,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-03 10:00:54,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-03 10:00:54,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-03 10:00:54,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-03 10:00:54,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-03 10:00:54,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-03 10:00:54,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-03 10:00:54,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-03 10:00:54,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-03 10:00:54,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-03 10:00:54,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-03 10:00:54,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-03 10:00:54,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-03 10:00:54,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-03 10:00:54,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-03 10:00:54,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-03 10:00:54,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-03 10:00:54,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-03 10:00:54,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-03 10:00:54,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-03 10:00:54,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-03 10:00:54,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-03 10:00:54,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-03 10:00:54,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-03 10:00:54,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-03 10:00:54,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-03 10:00:54,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-03 10:00:54,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-03 10:00:54,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-03 10:00:54,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-03 10:00:54,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-03 10:00:54,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-03 10:00:54,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-03 10:00:54,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-03 10:00:54,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-03 10:00:54,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-03 10:00:54,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-03 10:00:54,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-03 10:00:54,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-03 10:00:54,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-03 10:00:54,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-03 10:00:54,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-03 10:00:54,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-03 10:00:54,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-03 10:00:54,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-03 10:00:54,321 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03793326423113138, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05336488965426095, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03793326423113138, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17683333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24608333333333332, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-03 10:00:54,321 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-03 10:00:54,321 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18177609593216826, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03793326423113138, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05336488965426095, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03793326423113138, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17683333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24608333333333332, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-03 10:00:54,526 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 322 ms to try 5000 different iterations.  Found a solution that moves 2682 regions; Going from a computed imbalance of 0.18177609593216826 to a new imbalance of 0.013731254646893143. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.01915993554441384); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.029113347126710012, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.298, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.01915993554441384); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010833333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0415, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-03 10:00:54,534 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv433663735:76 , srv1180459024:77 , srv686494736:77 , srv1186985074:80 , srv1324114884:82 , srv1928447129:82 , srv54956873:82 , srv941975545:82 , srv127452002:83 , srv1291581568:83 , srv1532399695:83 , srv1554763571:83 , srv1625111152:83 , srv163064835:83 , srv1794965544:83 , srv269874159:83 , srv302658654:83 , srv519697047:83 , srv761822490:83 , srv1110416497:84 , srv1161486954:84 , srv119871269:84 , srv128851781:84 , srv1561948300:84 , srv187185342:84 , srv190183033:84 , srv1992999429:84 , srv2064505362:84 , srv2083015882:84 , srv330124450:84 , srv451743634:84 , srv546749091:84 , srv718788612:84 , srv94210750:84 , srv1003334649:85 , srv1029292851:85 , srv1185396553:85 , srv119080410:85 , srv1292040204:85 , srv1946383861:85 , srv1968697751:85 , srv1997051619:85 , srv472344242:85 , srv52189100:85 , srv643854097:85 , srv816639848:85 , srv989590250:85 , srv1201994079:86 , srv1328868920:86 , srv1353967962:86 , srv1429150942:86 , srv1595574430:86 , srv1684098858:86 , srv1740411852:86 , srv1760507329:86 , srv2082287730:86 , srv2104135761:86 , srv318056279:86 , srv605086189:86 , srv742183441:86 , srv812261468:86 , srv1032605757:87 , srv1348240354:87 , srv139167575:87 , srv1551005160:87 , srv1582988894:87 , srv1623230812:87 , srv1644305959:87 , srv177223491:87 , srv2119896556:87 , srv2138654190:87 , srv344887986:87 , srv515417680:87 , srv824580383:87 , srv1035555763:88 , srv1056075179:88 , srv111864724:88 , srv1141406499:88 , srv1273937045:88 , srv215258627:88 , srv589902059:88 , srv716520265:88 , srv1281343110:89 , srv1568881768:89 , srv19445090:89 , srv2036499128:89 , srv34277612:89 , srv404413399:89 , srv788817284:89 , srv969265360:89 , srv132893318:90 , srv1569726993:90 , srv1743190360:90 , srv1992490954:90 , srv2094322246:90 , srv947610687:90 , srv2131111515:91 , srv345717143:92 , srv92348499:93 , srv1542729188:517 }
2023-04-03 10:00:54,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-03 10:00:54,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv187185342=47, srv812261468=91, srv1032605757=2, srv1946383861=51, srv2036499128=56, srv1595574430=36, srv1328868920=22, srv969265360=98, srv1644305959=40, srv546749091=80, srv1760507329=44, srv2104135761=61, srv1185396553=10, srv1992999429=54, srv1568881768=33, srv54956873=81, srv1623230812=37, srv2082287730=58, srv1141406499=7, srv1554763571=31, srv215258627=65, srv1056075179=4, srv2119896556=62, srv1569726993=34, srv761822490=89, srv1740411852=42, srv1003334649=0, srv1180459024=9, srv1273937045=15, srv433663735=74, srv1743190360=43, srv111864724=6, srv92348499=94, srv119080410=12, srv1532399695=28, srv643854097=84, srv686494736=85, srv1281343110=17, srv1348240354=24, srv404413399=73, srv132893318=23, srv589902059=82, srv128851781=18, srv1201994079=14, srv605086189=83, srv302658654=67, srv1324114884=21, srv139167575=26, srv1684098858=41, srv1968697751=52, srv94210750=96, srv34277612=70, srv1794965544=46, srv1029292851=1, srv816639848=92, srv824580383=93, srv1992490954=53, srv19445090=50, srv2131111515=63, srv2138654190=64, srv344887986=71, srv788817284=90, srv515417680=77, srv318056279=68, srv2094322246=60, srv190183033=48, srv127452002=16, srv2064505362=57, srv472344242=76, srv269874159=66, srv718788612=87, srv1551005160=30, srv1997051619=55, srv345717143=72, srv119871269=13, srv1582988894=35, srv716520265=86, srv1542729188=29, srv1561948300=32, srv519697047=78, srv52189100=79, srv989590250=99, srv1625111152=38, srv742183441=88, srv1353967962=25, srv1110416497=5, srv1186985074=11, srv1291581568=19, srv177223491=45, srv330124450=69, srv163064835=39, srv2083015882=59, srv941975545=95, srv1928447129=49, srv1035555763=3, srv1429150942=27, srv947610687=97, srv1161486954=8, srv451743634=75, srv1292040204=20} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-03 10:00:54,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-03 10:00:54,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-03 10:00:54,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-03 10:00:54,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-03 10:00:54,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-03 10:00:54,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-03 10:00:54,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-03 10:00:54,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-03 10:00:54,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-03 10:00:54,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-03 10:00:54,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-03 10:00:54,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-03 10:00:54,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-03 10:00:54,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-03 10:00:54,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-03 10:00:54,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-03 10:00:54,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-03 10:00:54,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-03 10:00:54,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-03 10:00:54,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-03 10:00:54,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-03 10:00:54,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-03 10:00:54,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-03 10:00:54,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-03 10:00:54,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-03 10:00:54,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-03 10:00:54,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-03 10:00:54,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-03 10:00:54,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-03 10:00:54,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-03 10:00:54,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-03 10:00:54,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-03 10:00:54,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-03 10:00:54,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-03 10:00:54,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-03 10:00:54,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-03 10:00:54,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-03 10:00:54,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-03 10:00:54,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-03 10:00:54,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-03 10:00:54,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-03 10:00:54,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-03 10:00:54,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-03 10:00:54,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-03 10:00:54,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-03 10:00:54,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-03 10:00:54,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-03 10:00:54,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-03 10:00:54,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-03 10:00:54,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-03 10:00:54,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-03 10:00:54,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-03 10:00:54,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-03 10:00:54,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-03 10:00:54,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-03 10:00:54,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-03 10:00:54,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-03 10:00:54,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-03 10:00:54,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-03 10:00:54,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-03 10:00:54,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-03 10:00:54,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-03 10:00:54,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-03 10:00:54,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-03 10:00:54,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-03 10:00:54,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-03 10:00:54,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-03 10:00:54,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-03 10:00:54,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-03 10:00:54,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-03 10:00:54,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-03 10:00:54,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-03 10:00:54,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-03 10:00:54,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-03 10:00:54,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-03 10:00:54,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-03 10:00:54,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-03 10:00:54,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-03 10:00:54,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-03 10:00:54,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-03 10:00:54,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-03 10:00:54,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-03 10:00:54,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-03 10:00:54,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-03 10:00:54,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-03 10:00:54,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-03 10:00:54,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-03 10:00:54,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-03 10:00:54,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-03 10:00:54,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-03 10:00:54,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-03 10:00:54,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-03 10:00:54,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-03 10:00:54,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-03 10:00:54,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-03 10:00:54,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-03 10:00:54,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-03 10:00:54,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-03 10:00:54,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-03 10:00:54,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-03 10:00:54,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-03 10:00:54,647 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.01915993554441384); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.029113347126710012, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.01915993554441384); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010833333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0415, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-03 10:00:54,647 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-03 10:00:54,647 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.01371247488150517, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.01915993554441384); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.029113347126710012, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.01915993554441384); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010833333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0415, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-03 10:00:54,808 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 239 ms to try 5000 different iterations.  Found a solution that moves 968 regions; Going from a computed imbalance of 0.01371247488150517 to a new imbalance of 5.705744511746172E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005540945998106677); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007654139963827327); MoveCostFunction : (multiplier=7.0, imbalance=0.10755555555555556, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005540945998106677); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003916666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-03 10:00:54,810 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv345717143:81 , srv1032605757:82 , srv1743190360:82 , srv190183033:82 , srv2131111515:82 , srv215258627:82 , srv92348499:82 , srv111864724:83 , srv119080410:83 , srv1582988894:83 , srv1623230812:83 , srv1992490954:83 , srv1029292851:84 , srv1035555763:84 , srv127452002:84 , srv1281343110:84 , srv132893318:84 , srv1561948300:84 , srv1569726993:84 , srv1740411852:84 , srv2104135761:84 , srv302658654:84 , srv605086189:84 , srv643854097:84 , srv761822490:84 , srv788817284:84 , srv812261468:84 , srv947610687:84 , srv1003334649:85 , srv1110416497:85 , srv1141406499:85 , srv1180459024:85 , srv1185396553:85 , srv119871269:85 , srv1273937045:85 , srv1324114884:85 , srv1328868920:85 , srv1348240354:85 , srv1353967962:85 , srv139167575:85 , srv1554763571:85 , srv1568881768:85 , srv1644305959:85 , srv177223491:85 , srv1794965544:85 , srv1946383861:85 , srv2064505362:85 , srv2094322246:85 , srv2119896556:85 , srv2138654190:85 , srv34277612:85 , srv472344242:85 , srv52189100:85 , srv546749091:85 , srv54956873:85 , srv686494736:85 , srv716520265:85 , srv742183441:85 , srv1056075179:86 , srv1161486954:86 , srv1201994079:86 , srv1292040204:86 , srv1429150942:86 , srv1532399695:86 , srv1551005160:86 , srv1684098858:86 , srv19445090:86 , srv1968697751:86 , srv1992999429:86 , srv2036499128:86 , srv2082287730:86 , srv2083015882:86 , srv269874159:86 , srv330124450:86 , srv404413399:86 , srv519697047:86 , srv589902059:86 , srv718788612:86 , srv816639848:86 , srv824580383:86 , srv1186985074:87 , srv1595574430:87 , srv1625111152:87 , srv1760507329:87 , srv1928447129:87 , srv1997051619:87 , srv318056279:87 , srv344887986:87 , srv451743634:87 , srv515417680:87 , srv94210750:87 , srv128851781:88 , srv163064835:88 , srv187185342:88 , srv941975545:88 , srv969265360:88 , srv989590250:88 , srv1291581568:89 , srv433663735:89 , srv1542729188:563 }
2023-04-03 10:00:54,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-03 10:00:54,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv187185342=47, srv812261468=91, srv1032605757=2, srv1946383861=51, srv2036499128=56, srv1595574430=36, srv1328868920=22, srv969265360=98, srv1644305959=40, srv546749091=80, srv1760507329=44, srv2104135761=61, srv1185396553=10, srv1992999429=54, srv1568881768=33, srv54956873=81, srv1623230812=37, srv2082287730=58, srv1141406499=7, srv1554763571=31, srv215258627=65, srv1056075179=4, srv2119896556=62, srv1569726993=34, srv761822490=89, srv1740411852=42, srv1003334649=0, srv1180459024=9, srv1273937045=15, srv433663735=74, srv1743190360=43, srv111864724=6, srv92348499=94, srv119080410=12, srv1532399695=28, srv643854097=84, srv686494736=85, srv1281343110=17, srv1348240354=24, srv404413399=73, srv132893318=23, srv589902059=82, srv128851781=18, srv1201994079=14, srv605086189=83, srv302658654=67, srv1324114884=21, srv139167575=26, srv1684098858=41, srv1968697751=52, srv94210750=96, srv34277612=70, srv1794965544=46, srv1029292851=1, srv816639848=92, srv824580383=93, srv1992490954=53, srv19445090=50, srv2131111515=63, srv2138654190=64, srv344887986=71, srv788817284=90, srv515417680=77, srv318056279=68, srv2094322246=60, srv190183033=48, srv127452002=16, srv2064505362=57, srv472344242=76, srv269874159=66, srv718788612=87, srv1551005160=30, srv1997051619=55, srv345717143=72, srv119871269=13, srv1582988894=35, srv716520265=86, srv1542729188=29, srv1561948300=32, srv519697047=78, srv52189100=79, srv989590250=99, srv1625111152=38, srv742183441=88, srv1353967962=25, srv1110416497=5, srv1186985074=11, srv1291581568=19, srv177223491=45, srv330124450=69, srv163064835=39, srv2083015882=59, srv941975545=95, srv1928447129=49, srv1035555763=3, srv1429150942=27, srv947610687=97, srv1161486954=8, srv451743634=75, srv1292040204=20} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-03 10:00:54,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-03 10:00:54,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-03 10:00:54,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-03 10:00:54,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-03 10:00:54,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-03 10:00:54,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-03 10:00:54,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-03 10:00:54,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-03 10:00:54,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-03 10:00:54,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-03 10:00:54,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-03 10:00:54,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-03 10:00:54,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-03 10:00:54,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-03 10:00:54,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-03 10:00:54,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-03 10:00:54,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-03 10:00:54,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-03 10:00:54,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-03 10:00:54,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-03 10:00:54,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-03 10:00:54,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-03 10:00:54,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-03 10:00:54,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-03 10:00:54,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-03 10:00:54,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-03 10:00:54,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-03 10:00:54,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-03 10:00:54,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-03 10:00:54,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-03 10:00:54,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-03 10:00:54,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-03 10:00:54,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-03 10:00:54,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-03 10:00:54,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-03 10:00:54,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-03 10:00:54,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-03 10:00:54,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-03 10:00:54,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-03 10:00:54,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-03 10:00:54,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-03 10:00:54,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-03 10:00:54,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-03 10:00:54,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-03 10:00:54,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-03 10:00:54,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-03 10:00:54,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-03 10:00:54,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-03 10:00:54,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-03 10:00:54,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-03 10:00:54,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-03 10:00:54,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-03 10:00:54,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-03 10:00:54,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-03 10:00:54,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-03 10:00:54,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-03 10:00:54,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-03 10:00:54,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-03 10:00:54,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-03 10:00:54,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-03 10:00:54,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-03 10:00:54,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-03 10:00:54,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-03 10:00:54,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-03 10:00:54,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-03 10:00:54,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-03 10:00:54,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-03 10:00:54,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-03 10:00:54,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-03 10:00:54,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-03 10:00:54,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-03 10:00:54,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-03 10:00:54,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-03 10:00:54,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-03 10:00:54,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-03 10:00:54,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-03 10:00:54,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-03 10:00:54,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-03 10:00:54,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-03 10:00:54,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-03 10:00:54,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-03 10:00:54,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-03 10:00:54,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-03 10:00:54,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-03 10:00:54,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-03 10:00:54,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-03 10:00:54,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-03 10:00:54,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-03 10:00:54,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-03 10:00:54,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-03 10:00:54,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-03 10:00:54,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-03 10:00:54,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-03 10:00:54,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-03 10:00:54,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-03 10:00:54,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-03 10:00:54,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-03 10:00:54,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-03 10:00:54,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-03 10:00:54,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-03 10:00:54,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-03 10:00:54,949 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005540945998106677); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007654139963827327); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005540945998106677); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003916666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-03 10:00:54,949 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-03 10:00:54,949 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=5.637963703037899E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005540945998106677); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007654139963827327); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005540945998106677); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003916666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-03 10:00:55,198 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 344 ms to try 5000 different iterations.  Found a solution that moves 248 regions; Going from a computed imbalance of 5.637963703037899E-4 to a new imbalance of 1.9333642752301927E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017082358106351274); MoveCostFunction : (multiplier=7.0, imbalance=0.027555555555555555, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-03 10:00:55,199 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1110416497:83 , srv111864724:83 , srv1551005160:83 , srv1625111152:83 , srv177223491:83 , srv761822490:83 , srv824580383:83 , srv969265360:83 , srv1032605757:84 , srv1161486954:84 , srv1180459024:84 , srv1185396553:84 , srv1201994079:84 , srv1273937045:84 , srv128851781:84 , srv1291581568:84 , srv1324114884:84 , srv1353967962:84 , srv139167575:84 , srv1429150942:84 , srv1623230812:84 , srv163064835:84 , srv1740411852:84 , srv19445090:84 , srv1946383861:84 , srv1968697751:84 , srv2036499128:84 , srv2082287730:84 , srv2083015882:84 , srv2094322246:84 , srv2104135761:84 , srv2138654190:84 , srv215258627:84 , srv269874159:84 , srv318056279:84 , srv330124450:84 , srv433663735:84 , srv451743634:84 , srv546749091:84 , srv54956873:84 , srv589902059:84 , srv643854097:84 , srv718788612:84 , srv92348499:84 , srv941975545:84 , srv947610687:84 , srv989590250:84 , srv1003334649:85 , srv1029292851:85 , srv1035555763:85 , srv1056075179:85 , srv1141406499:85 , srv1186985074:85 , srv119871269:85 , srv127452002:85 , srv1281343110:85 , srv1292040204:85 , srv1328868920:85 , srv132893318:85 , srv1348240354:85 , srv1532399695:85 , srv1554763571:85 , srv1561948300:85 , srv1568881768:85 , srv1569726993:85 , srv1582988894:85 , srv1595574430:85 , srv1644305959:85 , srv1684098858:85 , srv1743190360:85 , srv1794965544:85 , srv187185342:85 , srv190183033:85 , srv1928447129:85 , srv1992490954:85 , srv1992999429:85 , srv1997051619:85 , srv2119896556:85 , srv2131111515:85 , srv34277612:85 , srv344887986:85 , srv345717143:85 , srv404413399:85 , srv472344242:85 , srv515417680:85 , srv519697047:85 , srv52189100:85 , srv686494736:85 , srv716520265:85 , srv742183441:85 , srv788817284:85 , srv812261468:85 , srv816639848:85 , srv94210750:85 , srv119080410:86 , srv1760507329:86 , srv2064505362:86 , srv302658654:86 , srv605086189:86 , srv1542729188:635 }
2023-04-03 10:00:55,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-03 10:00:55,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv187185342=47, srv812261468=91, srv1032605757=2, srv1946383861=51, srv2036499128=56, srv1595574430=36, srv1328868920=22, srv969265360=98, srv1644305959=40, srv546749091=80, srv1760507329=44, srv2104135761=61, srv1185396553=10, srv1992999429=54, srv1568881768=33, srv54956873=81, srv1623230812=37, srv2082287730=58, srv1141406499=7, srv1554763571=31, srv215258627=65, srv1056075179=4, srv2119896556=62, srv1569726993=34, srv761822490=89, srv1740411852=42, srv1003334649=0, srv1180459024=9, srv1273937045=15, srv433663735=74, srv1743190360=43, srv111864724=6, srv92348499=94, srv119080410=12, srv1532399695=28, srv643854097=84, srv686494736=85, srv1281343110=17, srv1348240354=24, srv404413399=73, srv132893318=23, srv589902059=82, srv128851781=18, srv1201994079=14, srv605086189=83, srv302658654=67, srv1324114884=21, srv139167575=26, srv1684098858=41, srv1968697751=52, srv94210750=96, srv34277612=70, srv1794965544=46, srv1029292851=1, srv816639848=92, srv824580383=93, srv1992490954=53, srv19445090=50, srv2131111515=63, srv2138654190=64, srv344887986=71, srv788817284=90, srv515417680=77, srv318056279=68, srv2094322246=60, srv190183033=48, srv127452002=16, srv2064505362=57, srv472344242=76, srv269874159=66, srv718788612=87, srv1551005160=30, srv1997051619=55, srv345717143=72, srv119871269=13, srv1582988894=35, srv716520265=86, srv1542729188=29, srv1561948300=32, srv519697047=78, srv52189100=79, srv989590250=99, srv1625111152=38, srv742183441=88, srv1353967962=25, srv1110416497=5, srv1186985074=11, srv1291581568=19, srv177223491=45, srv330124450=69, srv163064835=39, srv2083015882=59, srv941975545=95, srv1928447129=49, srv1035555763=3, srv1429150942=27, srv947610687=97, srv1161486954=8, srv451743634=75, srv1292040204=20} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-03 10:00:55,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-03 10:00:55,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-03 10:00:55,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-03 10:00:55,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-03 10:00:55,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-03 10:00:55,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-03 10:00:55,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-03 10:00:55,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-03 10:00:55,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-03 10:00:55,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-03 10:00:55,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-03 10:00:55,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-03 10:00:55,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-03 10:00:55,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-03 10:00:55,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-03 10:00:55,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-03 10:00:55,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-03 10:00:55,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-03 10:00:55,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-03 10:00:55,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-03 10:00:55,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-03 10:00:55,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-03 10:00:55,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-03 10:00:55,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-03 10:00:55,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-03 10:00:55,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-03 10:00:55,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-03 10:00:55,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-03 10:00:55,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-03 10:00:55,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-03 10:00:55,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-03 10:00:55,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-03 10:00:55,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-03 10:00:55,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-03 10:00:55,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-03 10:00:55,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-03 10:00:55,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-03 10:00:55,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-03 10:00:55,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-03 10:00:55,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-03 10:00:55,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-03 10:00:55,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-03 10:00:55,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-03 10:00:55,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-03 10:00:55,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-03 10:00:55,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-03 10:00:55,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-03 10:00:55,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-03 10:00:55,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-03 10:00:55,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-03 10:00:55,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-03 10:00:55,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-03 10:00:55,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-03 10:00:55,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-03 10:00:55,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-03 10:00:55,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-03 10:00:55,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-03 10:00:55,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-03 10:00:55,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-03 10:00:55,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-03 10:00:55,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-03 10:00:55,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-03 10:00:55,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-03 10:00:55,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-03 10:00:55,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-03 10:00:55,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-03 10:00:55,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-03 10:00:55,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-03 10:00:55,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-03 10:00:55,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-03 10:00:55,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-03 10:00:55,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-03 10:00:55,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-03 10:00:55,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-03 10:00:55,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-03 10:00:55,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-03 10:00:55,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-03 10:00:55,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-03 10:00:55,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-03 10:00:55,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-03 10:00:55,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-03 10:00:55,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-03 10:00:55,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-03 10:00:55,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-03 10:00:55,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-03 10:00:55,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-03 10:00:55,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-03 10:00:55,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-03 10:00:55,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-03 10:00:55,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-03 10:00:55,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-03 10:00:55,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-03 10:00:55,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-03 10:00:55,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-03 10:00:55,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-03 10:00:55,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-03 10:00:55,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-03 10:00:55,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-03 10:00:55,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-03 10:00:55,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-03 10:00:55,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-03 10:00:55,305 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.7597109636635417E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017082358106351274); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-03 10:00:55,305 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1110416497:83 , srv111864724:83 , srv1551005160:83 , srv1625111152:83 , srv177223491:83 , srv761822490:83 , srv824580383:83 , srv969265360:83 , srv1032605757:84 , srv1161486954:84 , srv1180459024:84 , srv1185396553:84 , srv1201994079:84 , srv1273937045:84 , srv128851781:84 , srv1291581568:84 , srv1324114884:84 , srv1353967962:84 , srv139167575:84 , srv1429150942:84 , srv1623230812:84 , srv163064835:84 , srv1740411852:84 , srv19445090:84 , srv1946383861:84 , srv1968697751:84 , srv2036499128:84 , srv2082287730:84 , srv2083015882:84 , srv2094322246:84 , srv2104135761:84 , srv2138654190:84 , srv215258627:84 , srv269874159:84 , srv318056279:84 , srv330124450:84 , srv433663735:84 , srv451743634:84 , srv546749091:84 , srv54956873:84 , srv589902059:84 , srv643854097:84 , srv718788612:84 , srv92348499:84 , srv941975545:84 , srv947610687:84 , srv989590250:84 , srv1003334649:85 , srv1029292851:85 , srv1035555763:85 , srv1056075179:85 , srv1141406499:85 , srv1186985074:85 , srv119871269:85 , srv127452002:85 , srv1281343110:85 , srv1292040204:85 , srv1328868920:85 , srv132893318:85 , srv1348240354:85 , srv1532399695:85 , srv1554763571:85 , srv1561948300:85 , srv1568881768:85 , srv1569726993:85 , srv1582988894:85 , srv1595574430:85 , srv1644305959:85 , srv1684098858:85 , srv1743190360:85 , srv1794965544:85 , srv187185342:85 , srv190183033:85 , srv1928447129:85 , srv1992490954:85 , srv1992999429:85 , srv1997051619:85 , srv2119896556:85 , srv2131111515:85 , srv34277612:85 , srv344887986:85 , srv345717143:85 , srv404413399:85 , srv472344242:85 , srv515417680:85 , srv519697047:85 , srv52189100:85 , srv686494736:85 , srv716520265:85 , srv742183441:85 , srv788817284:85 , srv812261468:85 , srv816639848:85 , srv94210750:85 , srv119080410:86 , srv1760507329:86 , srv2064505362:86 , srv302658654:86 , srv605086189:86 , srv1542729188:635 }
2023-04-03 10:00:55,327 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=319 (was 319), ProcessCount=166 (was 166), AvailableMemoryMB=4332 (was 4832)
2023-04-03 10:00:55,333 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=319, ProcessCount=166, AvailableMemoryMB=4332
2023-04-03 10:00:55,334 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-03 10:00:55,334 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-04-03 10:00:55,341 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1062933427:84 , srv1111199706:84 , srv1128476408:84 , srv1175782483:84 , srv1194617226:84 , srv1276082734:84 , srv1332565040:84 , srv143862097:84 , srv1470682303:84 , srv1502340568:84 , srv1613862811:84 , srv1679822736:84 , srv1726502047:84 , srv1784077137:84 , srv1876950092:84 , srv1880420890:84 , srv1903627432:84 , srv1919034083:84 , srv208807184:84 , srv411536006:84 , srv457244908:84 , srv538158744:84 , srv646362202:84 , srv695094610:84 , srv702403858:84 , srv836231897:84 , srv895978225:84 , srv931878527:84 , srv956461683:84 , srv1482653208:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-04-03 10:00:55,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-03 10:00:55,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1482653208=9, srv1502340568=10, srv695094610=24, srv1175782483=3, srv1679822736=12, srv1470682303=8, srv457244908=21, srv208807184=19, srv836231897=26, srv1111199706=1, srv956461683=29, srv1613862811=11, srv646362202=23, srv1332565040=6, srv1919034083=18, srv931878527=28, srv1903627432=17, srv1880420890=16, srv1128476408=2, srv143862097=7, srv1726502047=13, srv538158744=22, srv1784077137=14, srv1876950092=15, srv1194617226=4, srv1276082734=5, srv411536006=20, srv895978225=27, srv1062933427=0, srv702403858=25} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-03 10:00:55,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-03 10:00:55,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-03 10:00:55,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-03 10:00:55,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-03 10:00:55,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-03 10:00:55,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-03 10:00:55,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-03 10:00:55,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-03 10:00:55,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-03 10:00:55,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-03 10:00:55,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-03 10:00:55,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-03 10:00:55,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-03 10:00:55,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-03 10:00:55,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-03 10:00:55,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-03 10:00:55,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-03 10:00:55,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-03 10:00:55,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-03 10:00:55,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-03 10:00:55,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-03 10:00:55,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-03 10:00:55,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-03 10:00:55,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-03 10:00:55,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-03 10:00:55,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-03 10:00:55,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-03 10:00:55,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-03 10:00:55,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-03 10:00:55,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-03 10:00:55,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-04-03 10:00:55,364 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.099950911604211, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-03 10:00:55,364 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912507985323047, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.099950911604211, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-04-03 10:02:55,354 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 9234392 different iterations.  Found a solution that moves 1851 regions; Going from a computed imbalance of 0.9912507985323047 to a new imbalance of 4.320326340186438E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6855555555555556, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-03 10:02:55,359 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1062933427:90 , srv1111199706:90 , srv1128476408:90 , srv1175782483:90 , srv1194617226:90 , srv1276082734:90 , srv1332565040:90 , srv143862097:90 , srv1470682303:90 , srv1482653208:90 , srv1502340568:90 , srv1613862811:90 , srv1679822736:90 , srv1726502047:90 , srv1784077137:90 , srv1876950092:90 , srv1880420890:90 , srv1903627432:90 , srv1919034083:90 , srv208807184:90 , srv411536006:90 , srv457244908:90 , srv538158744:90 , srv646362202:90 , srv695094610:90 , srv702403858:90 , srv836231897:90 , srv895978225:90 , srv931878527:90 , srv956461683:90 }
2023-04-03 10:02:55,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-03 10:02:55,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1482653208=9, srv1502340568=10, srv695094610=24, srv1175782483=3, srv1679822736=12, srv1470682303=8, srv457244908=21, srv208807184=19, srv836231897=26, srv1111199706=1, srv956461683=29, srv1613862811=11, srv646362202=23, srv1332565040=6, srv1919034083=18, srv931878527=28, srv1903627432=17, srv1880420890=16, srv1128476408=2, srv143862097=7, srv1726502047=13, srv538158744=22, srv1784077137=14, srv1876950092=15, srv1194617226=4, srv1276082734=5, srv411536006=20, srv895978225=27, srv1062933427=0, srv702403858=25} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-03 10:02:55,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-03 10:02:55,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-03 10:02:55,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-03 10:02:55,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-03 10:02:55,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-03 10:02:55,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-03 10:02:55,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-03 10:02:55,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-03 10:02:55,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-03 10:02:55,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-03 10:02:55,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-03 10:02:55,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-03 10:02:55,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-03 10:02:55,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-03 10:02:55,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-03 10:02:55,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-03 10:02:55,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-03 10:02:55,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-03 10:02:55,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-03 10:02:55,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-03 10:02:55,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-03 10:02:55,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-03 10:02:55,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-03 10:02:55,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-03 10:02:55,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-03 10:02:55,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-03 10:02:55,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-03 10:02:55,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-03 10:02:55,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-03 10:02:55,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-03 10:02:55,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-04-03 10:02:55,374 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-03 10:02:55,374 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1062933427:90 , srv1111199706:90 , srv1128476408:90 , srv1175782483:90 , srv1194617226:90 , srv1276082734:90 , srv1332565040:90 , srv143862097:90 , srv1470682303:90 , srv1482653208:90 , srv1502340568:90 , srv1613862811:90 , srv1679822736:90 , srv1726502047:90 , srv1784077137:90 , srv1876950092:90 , srv1880420890:90 , srv1903627432:90 , srv1919034083:90 , srv208807184:90 , srv411536006:90 , srv457244908:90 , srv538158744:90 , srv646362202:90 , srv695094610:90 , srv702403858:90 , srv836231897:90 , srv895978225:90 , srv931878527:90 , srv956461683:90 }
2023-04-03 10:02:55,384 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=222 (was 319), ProcessCount=166 (was 166), AvailableMemoryMB=4222 (was 4332)