2023-05-04 05:53:33,782 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-05-04 05:53:34,126 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-05-04 05:53:34,244 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-05-04 05:53:34,309 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-04 05:53:34,319 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-05-04 05:53:34,335 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=354, ProcessCount=170, AvailableMemoryMB=7913
2023-05-04 05:53:34,337 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-04 05:53:34,338 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-05-04 05:53:34,468 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1007361823:84 , srv1015331921:84 , srv1055598675:84 , srv1071700070:84 , srv1077150450:84 , srv109171349:84 , srv1182496771:84 , srv1204200847:84 , srv1210044726:84 , srv1214223572:84 , srv1220826330:84 , srv1251515699:84 , srv1258574926:84 , srv1278674945:84 , srv1286074002:84 , srv1295909076:84 , srv1297670236:84 , srv1298816766:84 , srv1353661234:84 , srv1378792039:84 , srv1406156694:84 , srv1418086105:84 , srv1422362302:84 , srv1449789100:84 , srv1476793896:84 , srv1531407319:84 , srv1553325790:84 , srv1563207673:84 , srv1566682326:84 , srv1631347671:84 , srv1645599297:84 , srv1689903243:84 , srv1700018789:84 , srv1707639544:84 , srv1711670382:84 , srv1769984729:84 , srv1792121319:84 , srv1806742836:84 , srv1811466254:84 , srv1841239981:84 , srv1864958436:84 , srv191335099:84 , srv1924855736:84 , srv1927052904:84 , srv1932771611:84 , srv1939821465:84 , srv1943425758:84 , srv1976158922:84 , srv1988715651:84 , srv1992917027:84 , srv2051742803:84 , srv208202749:84 , srv2124575070:84 , srv2146754505:84 , srv2147247470:84 , srv24355920:84 , srv246435269:84 , srv257554037:84 , srv263959856:84 , srv270671970:84 , srv286509616:84 , srv286548047:84 , srv305219000:84 , srv323332143:84 , srv374090076:84 , srv394205772:84 , srv406289992:84 , srv432988438:84 , srv442490744:84 , srv453260780:84 , srv462800615:84 , srv480129799:84 , srv488468355:84 , srv503052464:84 , srv505937014:84 , srv539000722:84 , srv543729237:84 , srv564956001:84 , srv604607022:84 , srv635083970:84 , srv683881882:84 , srv705675483:84 , srv716984122:84 , srv736592686:84 , srv757719554:84 , srv815789793:84 , srv818787543:84 , srv821629048:84 , srv864159399:84 , srv887609140:84 , srv888949697:84 , srv905247465:84 , srv912046197:84 , srv915346379:84 , srv920694192:84 , srv943844982:84 , srv966994311:84 , srv982570935:84 , srv990791013:84 , srv916149720:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-05-04 05:53:34,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-04 05:53:34,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv912046197=92, srv543729237=76, srv1932771611=44, srv815789793=85, srv323332143=63, srv1406156694=20, srv1707639544=33, srv2147247470=54, srv1700018789=32, srv286548047=61, srv394205772=65, srv821629048=87, srv191335099=41, srv1689903243=31, srv24355920=55, srv757719554=84, srv1811466254=38, srv2051742803=50, srv887609140=89, srv920694192=95, srv1943425758=46, srv480129799=71, srv1077150450=4, srv635083970=79, srv990791013=99, srv1476793896=24, srv943844982=96, srv503052464=73, srv1645599297=30, srv1278674945=13, srv1939821465=45, srv305219000=62, srv1298816766=17, srv462800615=70, srv864159399=88, srv1976158922=47, srv2146754505=53, srv736592686=83, srv1378792039=19, srv1553325790=26, srv1711670382=34, srv286509616=60, srv1806742836=37, srv208202749=51, srv1286074002=14, srv1992917027=49, srv604607022=78, srv109171349=5, srv505937014=74, srv1841239981=39, srv564956001=77, srv1182496771=6, srv432988438=67, srv406289992=66, srv1792121319=36, srv1563207673=27, srv1007361823=0, srv488468355=72, srv966994311=97, srv1251515699=11, srv2124575070=52, srv1418086105=21, srv1071700070=3, srv818787543=86, srv1210044726=8, srv1015331921=1, srv1220826330=10, srv1297670236=16, srv263959856=58, srv1531407319=25, srv1353661234=18, srv1258574926=12, srv1988715651=48, srv374090076=64, srv1055598675=2, srv1769984729=35, srv257554037=57, srv270671970=59, srv705675483=81, srv1924855736=42, srv1295909076=15, srv1566682326=28, srv1631347671=29, srv1422362302=22, srv1864958436=40, srv905247465=91, srv888949697=90, srv539000722=75, srv982570935=98, srv1214223572=9, srv716984122=82, srv246435269=56, srv683881882=80, srv442490744=68, srv1449789100=23, srv915346379=93, srv916149720=94, srv1204200847=7, srv1927052904=43, srv453260780=69} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-04 05:53:34,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-04 05:53:34,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-04 05:53:34,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-04 05:53:34,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-04 05:53:34,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-04 05:53:34,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-04 05:53:34,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-04 05:53:34,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-04 05:53:34,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-04 05:53:34,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-04 05:53:34,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-04 05:53:34,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-04 05:53:34,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-04 05:53:34,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-04 05:53:34,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-04 05:53:34,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-04 05:53:34,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-04 05:53:34,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-04 05:53:34,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-04 05:53:34,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-04 05:53:34,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-04 05:53:34,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-04 05:53:34,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-04 05:53:34,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-04 05:53:34,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-04 05:53:34,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-04 05:53:34,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-04 05:53:34,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-04 05:53:34,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-04 05:53:34,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-04 05:53:34,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-04 05:53:34,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-04 05:53:34,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-04 05:53:34,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-04 05:53:34,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-04 05:53:34,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-04 05:53:34,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-04 05:53:34,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-04 05:53:34,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-04 05:53:34,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-04 05:53:34,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-04 05:53:34,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-04 05:53:34,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-04 05:53:34,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-04 05:53:34,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-04 05:53:34,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-04 05:53:34,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-04 05:53:34,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-04 05:53:34,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-04 05:53:34,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-04 05:53:34,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-04 05:53:34,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-04 05:53:34,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-04 05:53:34,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-04 05:53:34,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-04 05:53:34,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-04 05:53:34,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-04 05:53:34,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-04 05:53:34,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-04 05:53:34,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-04 05:53:34,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-04 05:53:34,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-04 05:53:34,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-04 05:53:34,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-04 05:53:34,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-04 05:53:34,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-04 05:53:34,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-04 05:53:34,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-04 05:53:34,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-04 05:53:34,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-04 05:53:34,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-04 05:53:34,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-04 05:53:34,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-04 05:53:34,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-04 05:53:34,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-04 05:53:34,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-04 05:53:34,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-04 05:53:34,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-04 05:53:34,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-04 05:53:34,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-04 05:53:34,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-04 05:53:34,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-04 05:53:34,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-04 05:53:34,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-04 05:53:34,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-04 05:53:34,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-04 05:53:34,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-04 05:53:34,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-04 05:53:34,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-04 05:53:34,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-04 05:53:34,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-04 05:53:34,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-04 05:53:34,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-04 05:53:34,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-04 05:53:34,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-04 05:53:34,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-04 05:53:34,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-04 05:53:34,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-04 05:53:34,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-04 05:53:34,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-04 05:53:34,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-04 05:53:34,892 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-04 05:53:34,892 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-04 05:53:34,893 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-04 05:53:35,287 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 658 ms to try 5000 different iterations.  Found a solution that moves 4681 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18243345745521503. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.041834417273782926, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05794459213400123, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5201111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.041834417273782926, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17725, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24841666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-04 05:53:35,312 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1278674945:82 , srv1932771611:82 , srv1939821465:82 , srv442490744:82 , srv1071700070:83 , srv109171349:83 , srv1182496771:83 , srv1298816766:83 , srv2051742803:83 , srv246435269:83 , srv305219000:83 , srv604607022:83 , srv716984122:83 , srv1258574926:84 , srv1700018789:84 , srv1806742836:84 , srv1976158922:84 , srv2146754505:84 , srv488468355:84 , srv683881882:84 , srv818787543:84 , srv905247465:84 , srv920694192:84 , srv966994311:84 , srv1007361823:85 , srv1077150450:85 , srv1210044726:85 , srv1220826330:85 , srv1378792039:85 , srv1418086105:85 , srv1531407319:85 , srv1563207673:85 , srv208202749:85 , srv2147247470:85 , srv270671970:85 , srv374090076:85 , srv432988438:85 , srv480129799:85 , srv815789793:85 , srv888949697:85 , srv915346379:85 , srv1055598675:86 , srv1297670236:86 , srv1422362302:86 , srv1711670382:86 , srv191335099:86 , srv2124575070:86 , srv24355920:86 , srv257554037:86 , srv286548047:86 , srv323332143:86 , srv406289992:86 , srv453260780:86 , srv462800615:86 , srv821629048:86 , srv990791013:86 , srv1015331921:87 , srv1204200847:87 , srv1214223572:87 , srv1566682326:87 , srv1707639544:87 , srv1769984729:87 , srv1811466254:87 , srv1864958436:87 , srv394205772:87 , srv505937014:87 , srv543729237:87 , srv635083970:87 , srv705675483:87 , srv757719554:87 , srv943844982:87 , srv1406156694:88 , srv1476793896:88 , srv1689903243:88 , srv1927052904:88 , srv263959856:88 , srv286509616:88 , srv564956001:88 , srv864159399:88 , srv887609140:88 , srv982570935:88 , srv1251515699:89 , srv1631347671:89 , srv1645599297:89 , srv1792121319:89 , srv1841239981:89 , srv1943425758:89 , srv1992917027:89 , srv503052464:89 , srv539000722:89 , srv912046197:89 , srv1449789100:90 , srv736592686:90 , srv1286074002:91 , srv1553325790:91 , srv1988715651:91 , srv1295909076:92 , srv1353661234:92 , srv1924855736:92 , srv916149720:462 }
2023-05-04 05:53:35,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-04 05:53:35,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv912046197=92, srv543729237=76, srv1932771611=44, srv815789793=85, srv323332143=63, srv1406156694=20, srv1707639544=33, srv2147247470=54, srv1700018789=32, srv286548047=61, srv394205772=65, srv821629048=87, srv191335099=41, srv1689903243=31, srv24355920=55, srv757719554=84, srv1811466254=38, srv2051742803=50, srv887609140=89, srv920694192=95, srv1943425758=46, srv480129799=71, srv1077150450=4, srv635083970=79, srv990791013=99, srv1476793896=24, srv943844982=96, srv503052464=73, srv1645599297=30, srv1278674945=13, srv1939821465=45, srv305219000=62, srv1298816766=17, srv462800615=70, srv864159399=88, srv1976158922=47, srv2146754505=53, srv736592686=83, srv1378792039=19, srv1553325790=26, srv1711670382=34, srv286509616=60, srv1806742836=37, srv208202749=51, srv1286074002=14, srv1992917027=49, srv604607022=78, srv109171349=5, srv505937014=74, srv1841239981=39, srv564956001=77, srv1182496771=6, srv432988438=67, srv406289992=66, srv1792121319=36, srv1563207673=27, srv1007361823=0, srv488468355=72, srv966994311=97, srv1251515699=11, srv2124575070=52, srv1418086105=21, srv1071700070=3, srv818787543=86, srv1210044726=8, srv1015331921=1, srv1220826330=10, srv1297670236=16, srv263959856=58, srv1531407319=25, srv1353661234=18, srv1258574926=12, srv1988715651=48, srv374090076=64, srv1055598675=2, srv1769984729=35, srv257554037=57, srv270671970=59, srv705675483=81, srv1924855736=42, srv1295909076=15, srv1566682326=28, srv1631347671=29, srv1422362302=22, srv1864958436=40, srv905247465=91, srv888949697=90, srv539000722=75, srv982570935=98, srv1214223572=9, srv716984122=82, srv246435269=56, srv683881882=80, srv442490744=68, srv1449789100=23, srv915346379=93, srv916149720=94, srv1204200847=7, srv1927052904=43, srv453260780=69} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-04 05:53:35,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-04 05:53:35,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-04 05:53:35,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-04 05:53:35,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-04 05:53:35,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-04 05:53:35,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-04 05:53:35,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-04 05:53:35,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-04 05:53:35,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-04 05:53:35,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-04 05:53:35,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-04 05:53:35,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-04 05:53:35,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-04 05:53:35,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-04 05:53:35,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-04 05:53:35,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-04 05:53:35,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-04 05:53:35,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-04 05:53:35,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-04 05:53:35,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-04 05:53:35,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-04 05:53:35,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-04 05:53:35,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-04 05:53:35,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-04 05:53:35,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-04 05:53:35,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-04 05:53:35,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-04 05:53:35,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-04 05:53:35,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-04 05:53:35,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-04 05:53:35,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-04 05:53:35,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-04 05:53:35,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-04 05:53:35,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-04 05:53:35,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-04 05:53:35,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-04 05:53:35,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-04 05:53:35,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-04 05:53:35,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-04 05:53:35,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-04 05:53:35,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-04 05:53:35,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-04 05:53:35,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-04 05:53:35,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-04 05:53:35,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-04 05:53:35,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-04 05:53:35,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-04 05:53:35,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-04 05:53:35,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-04 05:53:35,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-04 05:53:35,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-04 05:53:35,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-04 05:53:35,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-04 05:53:35,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-04 05:53:35,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-04 05:53:35,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-04 05:53:35,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-04 05:53:35,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-04 05:53:35,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-04 05:53:35,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-04 05:53:35,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-04 05:53:35,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-04 05:53:35,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-04 05:53:35,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-04 05:53:35,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-04 05:53:35,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-04 05:53:35,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-04 05:53:35,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-04 05:53:35,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-04 05:53:35,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-04 05:53:35,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-04 05:53:35,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-04 05:53:35,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-04 05:53:35,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-04 05:53:35,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-04 05:53:35,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-04 05:53:35,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-04 05:53:35,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-04 05:53:35,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-04 05:53:35,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-04 05:53:35,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-04 05:53:35,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-04 05:53:35,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-04 05:53:35,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-04 05:53:35,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-04 05:53:35,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-04 05:53:35,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-04 05:53:35,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-04 05:53:35,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-04 05:53:35,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-04 05:53:35,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-04 05:53:35,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-04 05:53:35,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-04 05:53:35,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-04 05:53:35,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-04 05:53:35,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-04 05:53:35,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-04 05:53:35,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-04 05:53:35,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-04 05:53:35,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-04 05:53:35,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-04 05:53:35,523 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.041834417273782926, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05794459213400123, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.041834417273782926, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17725, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24841666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-04 05:53:35,523 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-04 05:53:35,523 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18240068039265683, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.041834417273782926, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05794459213400123, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.041834417273782926, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17725, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24841666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-04 05:53:35,874 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 510 ms to try 5000 different iterations.  Found a solution that moves 2693 regions; Going from a computed imbalance of 0.18240068039265683 to a new imbalance of 0.016277038049251843. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020136706991212127); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.030273723201785348, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2992222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020136706991212127); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013416666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043833333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-04 05:53:35,886 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1295909076:78 , srv1353661234:79 , srv1553325790:80 , srv1988715651:80 , srv1286074002:81 , srv1924855736:81 , srv1943425758:82 , srv564956001:82 , srv1251515699:83 , srv1645599297:83 , srv1769984729:83 , srv1841239981:83 , srv263959856:83 , srv503052464:83 , srv736592686:83 , srv982570935:83 , srv1015331921:84 , srv1204200847:84 , srv1449789100:84 , srv1631347671:84 , srv1689903243:84 , srv1792121319:84 , srv1927052904:84 , srv1992917027:84 , srv286509616:84 , srv323332143:84 , srv394205772:84 , srv539000722:84 , srv635083970:84 , srv757719554:84 , srv912046197:84 , srv1055598675:85 , srv1811466254:85 , srv1864958436:85 , srv24355920:85 , srv462800615:85 , srv543729237:85 , srv1077150450:86 , srv1214223572:86 , srv1418086105:86 , srv1563207673:86 , srv1566682326:86 , srv1707639544:86 , srv191335099:86 , srv257554037:86 , srv270671970:86 , srv286548047:86 , srv432988438:86 , srv505937014:86 , srv864159399:86 , srv887609140:86 , srv990791013:86 , srv1007361823:87 , srv1071700070:87 , srv1182496771:87 , srv1297670236:87 , srv1406156694:87 , srv1422362302:87 , srv1476793896:87 , srv1531407319:87 , srv1711670382:87 , srv1806742836:87 , srv208202749:87 , srv2146754505:87 , srv2147247470:87 , srv374090076:87 , srv406289992:87 , srv480129799:87 , srv683881882:87 , srv705675483:87 , srv815789793:87 , srv818787543:87 , srv821629048:87 , srv888949697:87 , srv905247465:87 , srv943844982:87 , srv109171349:88 , srv1210044726:88 , srv1220826330:88 , srv1378792039:88 , srv2124575070:88 , srv453260780:88 , srv920694192:88 , srv966994311:88 , srv1258574926:89 , srv1298816766:89 , srv1976158922:89 , srv2051742803:89 , srv246435269:89 , srv305219000:89 , srv915346379:89 , srv1700018789:90 , srv1939821465:90 , srv442490744:90 , srv488468355:90 , srv716984122:90 , srv1932771611:91 , srv1278674945:93 , srv604607022:93 , srv916149720:491 }
2023-05-04 05:53:35,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-04 05:53:35,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv912046197=92, srv543729237=76, srv1932771611=44, srv815789793=85, srv323332143=63, srv1406156694=20, srv1707639544=33, srv2147247470=54, srv1700018789=32, srv286548047=61, srv394205772=65, srv821629048=87, srv191335099=41, srv1689903243=31, srv24355920=55, srv757719554=84, srv1811466254=38, srv2051742803=50, srv887609140=89, srv920694192=95, srv1943425758=46, srv480129799=71, srv1077150450=4, srv635083970=79, srv990791013=99, srv1476793896=24, srv943844982=96, srv503052464=73, srv1645599297=30, srv1278674945=13, srv1939821465=45, srv305219000=62, srv1298816766=17, srv462800615=70, srv864159399=88, srv1976158922=47, srv2146754505=53, srv736592686=83, srv1378792039=19, srv1553325790=26, srv1711670382=34, srv286509616=60, srv1806742836=37, srv208202749=51, srv1286074002=14, srv1992917027=49, srv604607022=78, srv109171349=5, srv505937014=74, srv1841239981=39, srv564956001=77, srv1182496771=6, srv432988438=67, srv406289992=66, srv1792121319=36, srv1563207673=27, srv1007361823=0, srv488468355=72, srv966994311=97, srv1251515699=11, srv2124575070=52, srv1418086105=21, srv1071700070=3, srv818787543=86, srv1210044726=8, srv1015331921=1, srv1220826330=10, srv1297670236=16, srv263959856=58, srv1531407319=25, srv1353661234=18, srv1258574926=12, srv1988715651=48, srv374090076=64, srv1055598675=2, srv1769984729=35, srv257554037=57, srv270671970=59, srv705675483=81, srv1924855736=42, srv1295909076=15, srv1566682326=28, srv1631347671=29, srv1422362302=22, srv1864958436=40, srv905247465=91, srv888949697=90, srv539000722=75, srv982570935=98, srv1214223572=9, srv716984122=82, srv246435269=56, srv683881882=80, srv442490744=68, srv1449789100=23, srv915346379=93, srv916149720=94, srv1204200847=7, srv1927052904=43, srv453260780=69} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-04 05:53:35,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-04 05:53:35,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-04 05:53:35,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-04 05:53:35,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-04 05:53:35,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-04 05:53:35,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-04 05:53:35,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-04 05:53:35,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-04 05:53:35,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-04 05:53:35,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-04 05:53:35,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-04 05:53:35,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-04 05:53:35,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-04 05:53:35,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-04 05:53:35,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-04 05:53:35,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-04 05:53:35,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-04 05:53:35,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-04 05:53:35,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-04 05:53:35,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-04 05:53:35,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-04 05:53:35,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-04 05:53:35,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-04 05:53:35,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-04 05:53:35,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-04 05:53:35,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-04 05:53:35,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-04 05:53:35,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-04 05:53:35,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-04 05:53:35,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-04 05:53:35,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-04 05:53:35,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-04 05:53:35,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-04 05:53:35,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-04 05:53:35,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-04 05:53:35,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-04 05:53:35,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-04 05:53:35,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-04 05:53:35,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-04 05:53:35,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-04 05:53:35,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-04 05:53:35,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-04 05:53:35,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-04 05:53:35,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-04 05:53:35,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-04 05:53:35,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-04 05:53:35,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-04 05:53:35,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-04 05:53:35,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-04 05:53:35,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-04 05:53:35,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-04 05:53:35,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-04 05:53:35,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-04 05:53:35,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-04 05:53:35,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-04 05:53:35,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-04 05:53:35,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-04 05:53:35,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-04 05:53:35,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-04 05:53:35,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-04 05:53:35,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-04 05:53:35,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-04 05:53:35,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-04 05:53:35,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-04 05:53:35,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-04 05:53:35,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-04 05:53:35,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-04 05:53:35,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-04 05:53:35,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-04 05:53:35,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-04 05:53:35,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-04 05:53:35,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-04 05:53:35,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-04 05:53:35,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-04 05:53:35,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-04 05:53:35,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-04 05:53:35,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-04 05:53:35,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-04 05:53:35,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-04 05:53:35,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-04 05:53:35,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-04 05:53:35,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-04 05:53:35,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-04 05:53:35,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-04 05:53:35,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-04 05:53:35,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-04 05:53:35,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-04 05:53:35,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-04 05:53:35,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-04 05:53:35,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-04 05:53:35,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-04 05:53:35,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-04 05:53:35,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-04 05:53:35,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-04 05:53:35,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-04 05:53:35,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-04 05:53:35,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-04 05:53:35,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-04 05:53:35,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-04 05:53:35,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-04 05:53:35,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-04 05:53:36,111 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020136706991212127); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.030273723201785348, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020136706991212127); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013416666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043833333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-04 05:53:36,111 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-04 05:53:36,111 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.016258181260217608, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020136706991212127); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.030273723201785348, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020136706991212127); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013416666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043833333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-04 05:53:36,471 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 531 ms to try 5000 different iterations.  Found a solution that moves 930 regions; Going from a computed imbalance of 0.016258181260217608 to a new imbalance of 7.076695133366199E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004976563103361181); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007106690545187014); MoveCostFunction : (multiplier=7.0, imbalance=0.10333333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.004976563103361181); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-04 05:53:36,473 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv604607022:81 , srv1278674945:83 , srv1406156694:83 , srv1476793896:83 , srv1700018789:83 , srv705675483:83 , srv915346379:83 , srv1210044726:84 , srv1214223572:84 , srv1220826330:84 , srv1258574926:84 , srv1422362302:84 , srv1449789100:84 , srv1711670382:84 , srv1924855736:84 , srv1932771611:84 , srv2124575070:84 , srv406289992:84 , srv453260780:84 , srv488468355:84 , srv505937014:84 , srv821629048:84 , srv887609140:84 , srv943844982:84 , srv1251515699:85 , srv1286074002:85 , srv1297670236:85 , srv1378792039:85 , srv1531407319:85 , srv1566682326:85 , srv1631347671:85 , srv1645599297:85 , srv1707639544:85 , srv1792121319:85 , srv1811466254:85 , srv1864958436:85 , srv191335099:85 , srv1939821465:85 , srv1992917027:85 , srv2051742803:85 , srv257554037:85 , srv286509616:85 , srv286548047:85 , srv503052464:85 , srv539000722:85 , srv716984122:85 , srv736592686:85 , srv815789793:85 , srv864159399:85 , srv888949697:85 , srv912046197:85 , srv966994311:85 , srv990791013:85 , srv1007361823:86 , srv1055598675:86 , srv1077150450:86 , srv1298816766:86 , srv1353661234:86 , srv1553325790:86 , srv1689903243:86 , srv1841239981:86 , srv1927052904:86 , srv1976158922:86 , srv208202749:86 , srv2146754505:86 , srv2147247470:86 , srv24355920:86 , srv246435269:86 , srv263959856:86 , srv305219000:86 , srv374090076:86 , srv432988438:86 , srv442490744:86 , srv462800615:86 , srv480129799:86 , srv543729237:86 , srv757719554:86 , srv818787543:86 , srv920694192:86 , srv982570935:86 , srv1015331921:87 , srv1071700070:87 , srv109171349:87 , srv1182496771:87 , srv1418086105:87 , srv1563207673:87 , srv1806742836:87 , srv1943425758:87 , srv1988715651:87 , srv270671970:87 , srv323332143:87 , srv394205772:87 , srv635083970:87 , srv683881882:87 , srv905247465:87 , srv1204200847:88 , srv1295909076:88 , srv1769984729:88 , srv564956001:88 , srv916149720:549 }
2023-05-04 05:53:36,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-04 05:53:36,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv912046197=92, srv543729237=76, srv1932771611=44, srv815789793=85, srv323332143=63, srv1406156694=20, srv1707639544=33, srv2147247470=54, srv1700018789=32, srv286548047=61, srv394205772=65, srv821629048=87, srv191335099=41, srv1689903243=31, srv24355920=55, srv757719554=84, srv1811466254=38, srv2051742803=50, srv887609140=89, srv920694192=95, srv1943425758=46, srv480129799=71, srv1077150450=4, srv635083970=79, srv990791013=99, srv1476793896=24, srv943844982=96, srv503052464=73, srv1645599297=30, srv1278674945=13, srv1939821465=45, srv305219000=62, srv1298816766=17, srv462800615=70, srv864159399=88, srv1976158922=47, srv2146754505=53, srv736592686=83, srv1378792039=19, srv1553325790=26, srv1711670382=34, srv286509616=60, srv1806742836=37, srv208202749=51, srv1286074002=14, srv1992917027=49, srv604607022=78, srv109171349=5, srv505937014=74, srv1841239981=39, srv564956001=77, srv1182496771=6, srv432988438=67, srv406289992=66, srv1792121319=36, srv1563207673=27, srv1007361823=0, srv488468355=72, srv966994311=97, srv1251515699=11, srv2124575070=52, srv1418086105=21, srv1071700070=3, srv818787543=86, srv1210044726=8, srv1015331921=1, srv1220826330=10, srv1297670236=16, srv263959856=58, srv1531407319=25, srv1353661234=18, srv1258574926=12, srv1988715651=48, srv374090076=64, srv1055598675=2, srv1769984729=35, srv257554037=57, srv270671970=59, srv705675483=81, srv1924855736=42, srv1295909076=15, srv1566682326=28, srv1631347671=29, srv1422362302=22, srv1864958436=40, srv905247465=91, srv888949697=90, srv539000722=75, srv982570935=98, srv1214223572=9, srv716984122=82, srv246435269=56, srv683881882=80, srv442490744=68, srv1449789100=23, srv915346379=93, srv916149720=94, srv1204200847=7, srv1927052904=43, srv453260780=69} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-04 05:53:36,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-04 05:53:36,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-04 05:53:36,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-04 05:53:36,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-04 05:53:36,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-04 05:53:36,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-04 05:53:36,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-04 05:53:36,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-04 05:53:36,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-04 05:53:36,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-04 05:53:36,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-04 05:53:36,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-04 05:53:36,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-04 05:53:36,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-04 05:53:36,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-04 05:53:36,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-04 05:53:36,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-04 05:53:36,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-04 05:53:36,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-04 05:53:36,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-04 05:53:36,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-04 05:53:36,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-04 05:53:36,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-04 05:53:36,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-04 05:53:36,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-04 05:53:36,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-04 05:53:36,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-04 05:53:36,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-04 05:53:36,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-04 05:53:36,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-04 05:53:36,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-04 05:53:36,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-04 05:53:36,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-04 05:53:36,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-04 05:53:36,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-04 05:53:36,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-04 05:53:36,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-04 05:53:36,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-04 05:53:36,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-04 05:53:36,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-04 05:53:36,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-04 05:53:36,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-04 05:53:36,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-04 05:53:36,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-04 05:53:36,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-04 05:53:36,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-04 05:53:36,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-04 05:53:36,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-04 05:53:36,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-04 05:53:36,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-04 05:53:36,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-04 05:53:36,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-04 05:53:36,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-04 05:53:36,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-04 05:53:36,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-04 05:53:36,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-04 05:53:36,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-04 05:53:36,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-04 05:53:36,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-04 05:53:36,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-04 05:53:36,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-04 05:53:36,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-04 05:53:36,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-04 05:53:36,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-04 05:53:36,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-04 05:53:36,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-04 05:53:36,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-04 05:53:36,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-04 05:53:36,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-04 05:53:36,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-04 05:53:36,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-04 05:53:36,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-04 05:53:36,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-04 05:53:36,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-04 05:53:36,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-04 05:53:36,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-04 05:53:36,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-04 05:53:36,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-04 05:53:36,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-04 05:53:36,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-04 05:53:36,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-04 05:53:36,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-04 05:53:36,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-04 05:53:36,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-04 05:53:36,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-04 05:53:36,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-04 05:53:36,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-04 05:53:36,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-04 05:53:36,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-04 05:53:36,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-04 05:53:36,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-04 05:53:36,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-04 05:53:36,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-04 05:53:36,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-04 05:53:36,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-04 05:53:36,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-04 05:53:36,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-04 05:53:36,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-04 05:53:36,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-04 05:53:36,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-04 05:53:36,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-04 05:53:36,619 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004976563103361181); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007106690545187014); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.004976563103361181); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-04 05:53:36,620 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-04 05:53:36,620 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=7.011575141528706E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004976563103361181); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007106690545187014); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.004976563103361181); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-04 05:53:36,997 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 482 ms to try 5000 different iterations.  Found a solution that moves 211 regions; Going from a computed imbalance of 7.011575141528706E-4 to a new imbalance of 3.39449909681629E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.5313429231318235E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.023444444444444445); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.5313429231318235E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-04 05:53:36,998 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1204200847:83 , srv864159399:83 , srv1007361823:84 , srv1055598675:84 , srv109171349:84 , srv1278674945:84 , srv1295909076:84 , srv1297670236:84 , srv1298816766:84 , srv1378792039:84 , srv1406156694:84 , srv1418086105:84 , srv1449789100:84 , srv1476793896:84 , srv1553325790:84 , srv1563207673:84 , srv1566682326:84 , srv1631347671:84 , srv1689903243:84 , srv1707639544:84 , srv1769984729:84 , srv1792121319:84 , srv1806742836:84 , srv1841239981:84 , srv1927052904:84 , srv1932771611:84 , srv1943425758:84 , srv1976158922:84 , srv1988715651:84 , srv1992917027:84 , srv2051742803:84 , srv208202749:84 , srv2124575070:84 , srv2147247470:84 , srv246435269:84 , srv270671970:84 , srv305219000:84 , srv374090076:84 , srv394205772:84 , srv442490744:84 , srv453260780:84 , srv462800615:84 , srv480129799:84 , srv488468355:84 , srv539000722:84 , srv543729237:84 , srv564956001:84 , srv635083970:84 , srv716984122:84 , srv736592686:84 , srv905247465:84 , srv912046197:84 , srv920694192:84 , srv943844982:84 , srv1015331921:85 , srv1071700070:85 , srv1077150450:85 , srv1182496771:85 , srv1210044726:85 , srv1214223572:85 , srv1220826330:85 , srv1251515699:85 , srv1258574926:85 , srv1286074002:85 , srv1353661234:85 , srv1422362302:85 , srv1531407319:85 , srv1645599297:85 , srv1700018789:85 , srv1711670382:85 , srv1811466254:85 , srv1864958436:85 , srv191335099:85 , srv1924855736:85 , srv1939821465:85 , srv2146754505:85 , srv24355920:85 , srv257554037:85 , srv263959856:85 , srv286509616:85 , srv286548047:85 , srv323332143:85 , srv406289992:85 , srv432988438:85 , srv503052464:85 , srv505937014:85 , srv604607022:85 , srv683881882:85 , srv705675483:85 , srv757719554:85 , srv815789793:85 , srv818787543:85 , srv821629048:85 , srv887609140:85 , srv888949697:85 , srv915346379:85 , srv966994311:85 , srv982570935:85 , srv990791013:86 , srv916149720:640 }
2023-05-04 05:53:37,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-04 05:53:37,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv912046197=92, srv543729237=76, srv1932771611=44, srv815789793=85, srv323332143=63, srv1406156694=20, srv1707639544=33, srv2147247470=54, srv1700018789=32, srv286548047=61, srv394205772=65, srv821629048=87, srv191335099=41, srv1689903243=31, srv24355920=55, srv757719554=84, srv1811466254=38, srv2051742803=50, srv887609140=89, srv920694192=95, srv1943425758=46, srv480129799=71, srv1077150450=4, srv635083970=79, srv990791013=99, srv1476793896=24, srv943844982=96, srv503052464=73, srv1645599297=30, srv1278674945=13, srv1939821465=45, srv305219000=62, srv1298816766=17, srv462800615=70, srv864159399=88, srv1976158922=47, srv2146754505=53, srv736592686=83, srv1378792039=19, srv1553325790=26, srv1711670382=34, srv286509616=60, srv1806742836=37, srv208202749=51, srv1286074002=14, srv1992917027=49, srv604607022=78, srv109171349=5, srv505937014=74, srv1841239981=39, srv564956001=77, srv1182496771=6, srv432988438=67, srv406289992=66, srv1792121319=36, srv1563207673=27, srv1007361823=0, srv488468355=72, srv966994311=97, srv1251515699=11, srv2124575070=52, srv1418086105=21, srv1071700070=3, srv818787543=86, srv1210044726=8, srv1015331921=1, srv1220826330=10, srv1297670236=16, srv263959856=58, srv1531407319=25, srv1353661234=18, srv1258574926=12, srv1988715651=48, srv374090076=64, srv1055598675=2, srv1769984729=35, srv257554037=57, srv270671970=59, srv705675483=81, srv1924855736=42, srv1295909076=15, srv1566682326=28, srv1631347671=29, srv1422362302=22, srv1864958436=40, srv905247465=91, srv888949697=90, srv539000722=75, srv982570935=98, srv1214223572=9, srv716984122=82, srv246435269=56, srv683881882=80, srv442490744=68, srv1449789100=23, srv915346379=93, srv916149720=94, srv1204200847=7, srv1927052904=43, srv453260780=69} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-04 05:53:37,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-04 05:53:37,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-04 05:53:37,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-04 05:53:37,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-04 05:53:37,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-04 05:53:37,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-04 05:53:37,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-04 05:53:37,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-04 05:53:37,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-04 05:53:37,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-04 05:53:37,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-04 05:53:37,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-04 05:53:37,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-04 05:53:37,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-04 05:53:37,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-04 05:53:37,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-04 05:53:37,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-04 05:53:37,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-04 05:53:37,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-04 05:53:37,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-04 05:53:37,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-04 05:53:37,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-04 05:53:37,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-04 05:53:37,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-04 05:53:37,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-04 05:53:37,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-04 05:53:37,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-04 05:53:37,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-04 05:53:37,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-04 05:53:37,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-04 05:53:37,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-04 05:53:37,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-04 05:53:37,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-04 05:53:37,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-04 05:53:37,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-04 05:53:37,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-04 05:53:37,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-04 05:53:37,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-04 05:53:37,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-04 05:53:37,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-04 05:53:37,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-04 05:53:37,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-04 05:53:37,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-04 05:53:37,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-04 05:53:37,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-04 05:53:37,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-04 05:53:37,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-04 05:53:37,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-04 05:53:37,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-04 05:53:37,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-04 05:53:37,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-04 05:53:37,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-04 05:53:37,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-04 05:53:37,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-04 05:53:37,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-04 05:53:37,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-04 05:53:37,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-04 05:53:37,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-04 05:53:37,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-04 05:53:37,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-04 05:53:37,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-04 05:53:37,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-04 05:53:37,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-04 05:53:37,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-04 05:53:37,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-04 05:53:37,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-04 05:53:37,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-04 05:53:37,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-04 05:53:37,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-04 05:53:37,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-04 05:53:37,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-04 05:53:37,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-04 05:53:37,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-04 05:53:37,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-04 05:53:37,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-04 05:53:37,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-04 05:53:37,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-04 05:53:37,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-04 05:53:37,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-04 05:53:37,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-04 05:53:37,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-04 05:53:37,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-04 05:53:37,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-04 05:53:37,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-04 05:53:37,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-04 05:53:37,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-04 05:53:37,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-04 05:53:37,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-04 05:53:37,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-04 05:53:37,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-04 05:53:37,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-04 05:53:37,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-04 05:53:37,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-04 05:53:37,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-04 05:53:37,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-04 05:53:37,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-04 05:53:37,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-04 05:53:37,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-04 05:53:37,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-04 05:53:37,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-04 05:53:37,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-04 05:53:37,126 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.5313429231318235E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.5313429231318235E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-04 05:53:37,126 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-04 05:53:37,127 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=3.246753738991438E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.5313429231318235E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.5313429231318235E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-04 05:53:37,431 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 398 ms to try 5000 different iterations.  Found a solution that moves 20 regions; Going from a computed imbalance of 3.246753738991438E-5 to a new imbalance of 7.213282289528359E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0015713484026367722); MoveCostFunction : (multiplier=7.0, imbalance=0.0022222222222222222); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-04 05:53:37,431 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1015331921:83 , srv1976158922:83 , srv683881882:83 , srv887609140:83 , srv990791013:83 , srv1007361823:84 , srv1071700070:84 , srv1077150450:84 , srv109171349:84 , srv1182496771:84 , srv1204200847:84 , srv1210044726:84 , srv1214223572:84 , srv1220826330:84 , srv1251515699:84 , srv1258574926:84 , srv1278674945:84 , srv1286074002:84 , srv1295909076:84 , srv1297670236:84 , srv1298816766:84 , srv1353661234:84 , srv1378792039:84 , srv1406156694:84 , srv1418086105:84 , srv1422362302:84 , srv1449789100:84 , srv1476793896:84 , srv1531407319:84 , srv1563207673:84 , srv1566682326:84 , srv1631347671:84 , srv1645599297:84 , srv1689903243:84 , srv1700018789:84 , srv1707639544:84 , srv1711670382:84 , srv1769984729:84 , srv1792121319:84 , srv1806742836:84 , srv1811466254:84 , srv1841239981:84 , srv1864958436:84 , srv191335099:84 , srv1924855736:84 , srv1927052904:84 , srv1939821465:84 , srv1943425758:84 , srv1988715651:84 , srv1992917027:84 , srv208202749:84 , srv2124575070:84 , srv2146754505:84 , srv2147247470:84 , srv24355920:84 , srv246435269:84 , srv257554037:84 , srv263959856:84 , srv270671970:84 , srv286509616:84 , srv286548047:84 , srv305219000:84 , srv323332143:84 , srv374090076:84 , srv394205772:84 , srv406289992:84 , srv432988438:84 , srv442490744:84 , srv453260780:84 , srv480129799:84 , srv488468355:84 , srv503052464:84 , srv505937014:84 , srv539000722:84 , srv543729237:84 , srv564956001:84 , srv604607022:84 , srv635083970:84 , srv705675483:84 , srv716984122:84 , srv736592686:84 , srv757719554:84 , srv815789793:84 , srv818787543:84 , srv821629048:84 , srv864159399:84 , srv888949697:84 , srv905247465:84 , srv912046197:84 , srv915346379:84 , srv920694192:84 , srv943844982:84 , srv966994311:84 , srv982570935:84 , srv1055598675:85 , srv1553325790:85 , srv1932771611:85 , srv2051742803:85 , srv462800615:85 , srv916149720:684 }
2023-05-04 05:53:37,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-04 05:53:37,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv912046197=92, srv543729237=76, srv1932771611=44, srv815789793=85, srv323332143=63, srv1406156694=20, srv1707639544=33, srv2147247470=54, srv1700018789=32, srv286548047=61, srv394205772=65, srv821629048=87, srv191335099=41, srv1689903243=31, srv24355920=55, srv757719554=84, srv1811466254=38, srv2051742803=50, srv887609140=89, srv920694192=95, srv1943425758=46, srv480129799=71, srv1077150450=4, srv635083970=79, srv990791013=99, srv1476793896=24, srv943844982=96, srv503052464=73, srv1645599297=30, srv1278674945=13, srv1939821465=45, srv305219000=62, srv1298816766=17, srv462800615=70, srv864159399=88, srv1976158922=47, srv2146754505=53, srv736592686=83, srv1378792039=19, srv1553325790=26, srv1711670382=34, srv286509616=60, srv1806742836=37, srv208202749=51, srv1286074002=14, srv1992917027=49, srv604607022=78, srv109171349=5, srv505937014=74, srv1841239981=39, srv564956001=77, srv1182496771=6, srv432988438=67, srv406289992=66, srv1792121319=36, srv1563207673=27, srv1007361823=0, srv488468355=72, srv966994311=97, srv1251515699=11, srv2124575070=52, srv1418086105=21, srv1071700070=3, srv818787543=86, srv1210044726=8, srv1015331921=1, srv1220826330=10, srv1297670236=16, srv263959856=58, srv1531407319=25, srv1353661234=18, srv1258574926=12, srv1988715651=48, srv374090076=64, srv1055598675=2, srv1769984729=35, srv257554037=57, srv270671970=59, srv705675483=81, srv1924855736=42, srv1295909076=15, srv1566682326=28, srv1631347671=29, srv1422362302=22, srv1864958436=40, srv905247465=91, srv888949697=90, srv539000722=75, srv982570935=98, srv1214223572=9, srv716984122=82, srv246435269=56, srv683881882=80, srv442490744=68, srv1449789100=23, srv915346379=93, srv916149720=94, srv1204200847=7, srv1927052904=43, srv453260780=69} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-04 05:53:37,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-04 05:53:37,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-04 05:53:37,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-04 05:53:37,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-04 05:53:37,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-04 05:53:37,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-04 05:53:37,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-04 05:53:37,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-04 05:53:37,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-04 05:53:37,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-04 05:53:37,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-04 05:53:37,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-04 05:53:37,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-04 05:53:37,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-04 05:53:37,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-04 05:53:37,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-04 05:53:37,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-04 05:53:37,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-04 05:53:37,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-04 05:53:37,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-04 05:53:37,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-04 05:53:37,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-04 05:53:37,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-04 05:53:37,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-04 05:53:37,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-04 05:53:37,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-04 05:53:37,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-04 05:53:37,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-04 05:53:37,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-04 05:53:37,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-04 05:53:37,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-04 05:53:37,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-04 05:53:37,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-04 05:53:37,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-04 05:53:37,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-04 05:53:37,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-04 05:53:37,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-04 05:53:37,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-04 05:53:37,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-04 05:53:37,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-04 05:53:37,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-04 05:53:37,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-04 05:53:37,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-04 05:53:37,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-04 05:53:37,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-04 05:53:37,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-04 05:53:37,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-04 05:53:37,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-04 05:53:37,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-04 05:53:37,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-04 05:53:37,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-04 05:53:37,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-04 05:53:37,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-04 05:53:37,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-04 05:53:37,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-04 05:53:37,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-04 05:53:37,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-04 05:53:37,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-04 05:53:37,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-04 05:53:37,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-04 05:53:37,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-04 05:53:37,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-04 05:53:37,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-04 05:53:37,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-04 05:53:37,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-04 05:53:37,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-04 05:53:37,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-04 05:53:37,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-04 05:53:37,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-04 05:53:37,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-04 05:53:37,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-04 05:53:37,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-04 05:53:37,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-04 05:53:37,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-04 05:53:37,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-04 05:53:37,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-04 05:53:37,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-04 05:53:37,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-04 05:53:37,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-04 05:53:37,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-04 05:53:37,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-04 05:53:37,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-04 05:53:37,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-04 05:53:37,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-04 05:53:37,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-04 05:53:37,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-04 05:53:37,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-04 05:53:37,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-04 05:53:37,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-04 05:53:37,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-04 05:53:37,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-04 05:53:37,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-04 05:53:37,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-04 05:53:37,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-04 05:53:37,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-04 05:53:37,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-04 05:53:37,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-04 05:53:37,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-04 05:53:37,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-04 05:53:37,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-04 05:53:37,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-04 05:53:37,574 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=7.073239296329447E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0015713484026367722); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-04 05:53:37,575 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1015331921:83 , srv1976158922:83 , srv683881882:83 , srv887609140:83 , srv990791013:83 , srv1007361823:84 , srv1071700070:84 , srv1077150450:84 , srv109171349:84 , srv1182496771:84 , srv1204200847:84 , srv1210044726:84 , srv1214223572:84 , srv1220826330:84 , srv1251515699:84 , srv1258574926:84 , srv1278674945:84 , srv1286074002:84 , srv1295909076:84 , srv1297670236:84 , srv1298816766:84 , srv1353661234:84 , srv1378792039:84 , srv1406156694:84 , srv1418086105:84 , srv1422362302:84 , srv1449789100:84 , srv1476793896:84 , srv1531407319:84 , srv1563207673:84 , srv1566682326:84 , srv1631347671:84 , srv1645599297:84 , srv1689903243:84 , srv1700018789:84 , srv1707639544:84 , srv1711670382:84 , srv1769984729:84 , srv1792121319:84 , srv1806742836:84 , srv1811466254:84 , srv1841239981:84 , srv1864958436:84 , srv191335099:84 , srv1924855736:84 , srv1927052904:84 , srv1939821465:84 , srv1943425758:84 , srv1988715651:84 , srv1992917027:84 , srv208202749:84 , srv2124575070:84 , srv2146754505:84 , srv2147247470:84 , srv24355920:84 , srv246435269:84 , srv257554037:84 , srv263959856:84 , srv270671970:84 , srv286509616:84 , srv286548047:84 , srv305219000:84 , srv323332143:84 , srv374090076:84 , srv394205772:84 , srv406289992:84 , srv432988438:84 , srv442490744:84 , srv453260780:84 , srv480129799:84 , srv488468355:84 , srv503052464:84 , srv505937014:84 , srv539000722:84 , srv543729237:84 , srv564956001:84 , srv604607022:84 , srv635083970:84 , srv705675483:84 , srv716984122:84 , srv736592686:84 , srv757719554:84 , srv815789793:84 , srv818787543:84 , srv821629048:84 , srv864159399:84 , srv888949697:84 , srv905247465:84 , srv912046197:84 , srv915346379:84 , srv920694192:84 , srv943844982:84 , srv966994311:84 , srv982570935:84 , srv1055598675:85 , srv1553325790:85 , srv1932771611:85 , srv2051742803:85 , srv462800615:85 , srv916149720:684 }
2023-05-04 05:53:37,606 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=358 (was 354) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=7034 (was 7913)
2023-05-04 05:53:37,623 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=358, ProcessCount=170, AvailableMemoryMB=7032
2023-05-04 05:53:37,624 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-04 05:53:37,624 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-05-04 05:53:37,636 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1084855256:84 , srv1091294167:84 , srv1134793187:84 , srv1183694749:84 , srv1207284446:84 , srv1260871125:84 , srv1276902480:84 , srv1309362505:84 , srv1380916740:84 , srv1396342424:84 , srv1411281058:84 , srv1451050804:84 , srv145132695:84 , srv1458173761:84 , srv1869546926:84 , srv192996346:84 , srv2013729040:84 , srv2039307108:84 , srv2077610969:84 , srv2079437165:84 , srv2135316888:84 , srv237538270:84 , srv285701673:84 , srv347974598:84 , srv436557797:84 , srv45622188:84 , srv492616865:84 , srv673435329:84 , srv808965541:84 , srv653952493:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-05-04 05:53:37,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-04 05:53:37,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv808965541=29, srv1411281058=10, srv492616865=26, srv653952493=27, srv1134793187=2, srv2077610969=18, srv1458173761=13, srv673435329=28, srv1260871125=5, srv1084855256=0, srv45622188=25, srv2135316888=20, srv1869546926=14, srv1183694749=3, srv436557797=24, srv1207284446=4, srv1396342424=9, srv145132695=12, srv192996346=15, srv347974598=23, srv1309362505=7, srv1380916740=8, srv2039307108=17, srv285701673=22, srv1091294167=1, srv1276902480=6, srv1451050804=11, srv2079437165=19, srv2013729040=16, srv237538270=21} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-04 05:53:37,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-04 05:53:37,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-04 05:53:37,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-04 05:53:37,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-04 05:53:37,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-04 05:53:37,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-04 05:53:37,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-04 05:53:37,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-04 05:53:37,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-04 05:53:37,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-04 05:53:37,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-04 05:53:37,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-04 05:53:37,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-04 05:53:37,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-04 05:53:37,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-04 05:53:37,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-04 05:53:37,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-04 05:53:37,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-04 05:53:37,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-04 05:53:37,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-04 05:53:37,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-04 05:53:37,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-04 05:53:37,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-04 05:53:37,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-04 05:53:37,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-04 05:53:37,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-04 05:53:37,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-04 05:53:37,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-04 05:53:37,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-04 05:53:37,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-04 05:53:37,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-05-04 05:53:37,681 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1111562210122696, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-04 05:53:37,682 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912543292887105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1111562210122696, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-05-04 05:55:37,660 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6140222 different iterations.  Found a solution that moves 1852 regions; Going from a computed imbalance of 0.9912543292887105 to a new imbalance of 4.322660390073086E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6859259259259259, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-04 05:55:37,666 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1084855256:90 , srv1091294167:90 , srv1134793187:90 , srv1183694749:90 , srv1207284446:90 , srv1260871125:90 , srv1276902480:90 , srv1309362505:90 , srv1380916740:90 , srv1396342424:90 , srv1411281058:90 , srv1451050804:90 , srv145132695:90 , srv1458173761:90 , srv1869546926:90 , srv192996346:90 , srv2013729040:90 , srv2039307108:90 , srv2077610969:90 , srv2079437165:90 , srv2135316888:90 , srv237538270:90 , srv285701673:90 , srv347974598:90 , srv436557797:90 , srv45622188:90 , srv492616865:90 , srv653952493:90 , srv673435329:90 , srv808965541:90 }
2023-05-04 05:55:37,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-04 05:55:37,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv808965541=29, srv1411281058=10, srv492616865=26, srv653952493=27, srv1134793187=2, srv2077610969=18, srv1458173761=13, srv673435329=28, srv1260871125=5, srv1084855256=0, srv45622188=25, srv2135316888=20, srv1869546926=14, srv1183694749=3, srv436557797=24, srv1207284446=4, srv1396342424=9, srv145132695=12, srv192996346=15, srv347974598=23, srv1309362505=7, srv1380916740=8, srv2039307108=17, srv285701673=22, srv1091294167=1, srv1276902480=6, srv1451050804=11, srv2079437165=19, srv2013729040=16, srv237538270=21} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-04 05:55:37,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-04 05:55:37,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-04 05:55:37,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-04 05:55:37,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-04 05:55:37,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-04 05:55:37,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-04 05:55:37,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-04 05:55:37,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-04 05:55:37,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-04 05:55:37,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-04 05:55:37,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-04 05:55:37,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-04 05:55:37,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-04 05:55:37,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-04 05:55:37,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-04 05:55:37,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-04 05:55:37,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-04 05:55:37,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-04 05:55:37,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-04 05:55:37,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-04 05:55:37,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-04 05:55:37,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-04 05:55:37,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-04 05:55:37,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-04 05:55:37,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-04 05:55:37,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-04 05:55:37,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-04 05:55:37,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-04 05:55:37,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-04 05:55:37,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-04 05:55:37,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-05-04 05:55:37,692 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-04 05:55:37,692 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1084855256:90 , srv1091294167:90 , srv1134793187:90 , srv1183694749:90 , srv1207284446:90 , srv1260871125:90 , srv1276902480:90 , srv1309362505:90 , srv1380916740:90 , srv1396342424:90 , srv1411281058:90 , srv1451050804:90 , srv145132695:90 , srv1458173761:90 , srv1869546926:90 , srv192996346:90 , srv2013729040:90 , srv2039307108:90 , srv2077610969:90 , srv2079437165:90 , srv2135316888:90 , srv237538270:90 , srv285701673:90 , srv347974598:90 , srv436557797:90 , srv45622188:90 , srv492616865:90 , srv653952493:90 , srv673435329:90 , srv808965541:90 }
2023-05-04 05:55:37,710 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=191 (was 358), ProcessCount=170 (was 170), AvailableMemoryMB=7145 (was 7032) - AvailableMemoryMB LEAK? -