2023-09-18 07:53:11,273 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-09-18 07:53:11,596 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-09-18 07:53:11,709 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-09-18 07:53:11,776 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-09-18 07:53:11,785 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-09-18 07:53:11,798 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=248, ProcessCount=169, AvailableMemoryMB=10005
2023-09-18 07:53:11,800 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-09-18 07:53:11,800 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-09-18 07:53:11,957 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1002936450:84 , srv1026722663:84 , srv104160899:84 , srv1042408835:84 , srv1045914756:84 , srv1099009312:84 , srv112507680:84 , srv1192972907:84 , srv1244466770:84 , srv1263920127:84 , srv1290934995:84 , srv1331808098:84 , srv1347387609:84 , srv1348242334:84 , srv1363664170:84 , srv1364823596:84 , srv1364997080:84 , srv1389836630:84 , srv1393368018:84 , srv1404540767:84 , srv1419437673:84 , srv1441765435:84 , srv1442102759:84 , srv1450649307:84 , srv1467231617:84 , srv1484667056:84 , srv1487613277:84 , srv1504990062:84 , srv1505992907:84 , srv1514604712:84 , srv1515621783:84 , srv1515645687:84 , srv1519024577:84 , srv1533169314:84 , srv1556642375:84 , srv1563463730:84 , srv160456350:84 , srv1655294863:84 , srv1655772079:84 , srv1657210050:84 , srv1692716423:84 , srv169533209:84 , srv1709239806:84 , srv1723157668:84 , srv1753751030:84 , srv1773315984:84 , srv1781720329:84 , srv1806257125:84 , srv1949043756:84 , srv195514735:84 , srv1964425110:84 , srv1976474890:84 , srv2007910844:84 , srv2026875976:84 , srv2070956033:84 , srv2079952052:84 , srv2100108773:84 , srv2102197120:84 , srv210886203:84 , srv211529347:84 , srv2127720055:84 , srv2139413132:84 , srv2140697095:84 , srv217006633:84 , srv24078967:84 , srv298866798:84 , srv304711778:84 , srv308191181:84 , srv319101941:84 , srv337674047:84 , srv344646834:84 , srv413290954:84 , srv421726031:84 , srv458237854:84 , srv462554789:84 , srv481685376:84 , srv51299728:84 , srv543683809:84 , srv558618581:84 , srv59984462:84 , srv62269457:84 , srv624909926:84 , srv629492933:84 , srv652413458:84 , srv653365683:84 , srv653383198:84 , srv668072159:84 , srv704555886:84 , srv796673089:84 , srv811250498:84 , srv813531251:84 , srv831312246:84 , srv834842376:84 , srv866842406:84 , srv881899638:84 , srv896852993:84 , srv91851259:84 , srv964686832:84 , srv995813261:84 , srv1103296073:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-09-18 07:53:11,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-18 07:53:11,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2127720055=61, srv1773315984=46, srv834842376=93, srv51299728=77, srv24078967=65, srv704555886=88, srv413290954=72, srv1331808098=12, srv1709239806=43, srv481685376=76, srv458237854=74, srv1533169314=34, srv813531251=91, srv1519024577=33, srv896852993=96, srv1419437673=21, srv2070956033=55, srv211529347=60, srv1450649307=24, srv421726031=73, srv831312246=92, srv1487613277=27, srv62269457=81, srv1504990062=28, srv1364823596=16, srv1045914756=4, srv629492933=83, srv1404540767=20, srv2079952052=56, srv59984462=80, srv1389836630=18, srv1556642375=35, srv304711778=67, srv344646834=71, srv1042408835=3, srv1964425110=51, srv169533209=42, srv1781720329=47, srv195514735=50, srv1192972907=8, srv1655772079=39, srv1563463730=36, srv319101941=69, srv1099009312=5, srv2102197120=58, srv210886203=59, srv298866798=66, srv1753751030=45, srv1002936450=0, srv624909926=82, srv462554789=75, srv160456350=37, srv2026875976=54, srv1026722663=1, srv112507680=7, srv337674047=70, srv1723157668=44, srv1515621783=31, srv653383198=86, srv1514604712=30, srv881899638=95, srv1363664170=15, srv964686832=98, srv2007910844=53, srv1364997080=17, srv1467231617=25, srv2100108773=57, srv1441765435=22, srv1949043756=49, srv1244466770=9, srv104160899=2, srv1692716423=41, srv866842406=94, srv1263920127=10, srv1655294863=38, srv1505992907=29, srv652413458=84, srv1442102759=23, srv1484667056=26, srv1657210050=40, srv1806257125=48, srv91851259=97, srv811250498=90, srv1347387609=13, srv1393368018=19, srv1103296073=6, srv668072159=87, srv543683809=78, srv558618581=79, srv1976474890=52, srv1515645687=32, srv308191181=68, srv1290934995=11, srv2139413132=62, srv1348242334=14, srv217006633=64, srv995813261=99, srv2140697095=63, srv796673089=89, srv653365683=85} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-18 07:53:12,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-18 07:53:12,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-18 07:53:12,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-18 07:53:12,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-18 07:53:12,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-18 07:53:12,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-18 07:53:12,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-18 07:53:12,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-18 07:53:12,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-18 07:53:12,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-18 07:53:12,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-18 07:53:12,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-18 07:53:12,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-18 07:53:12,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-18 07:53:12,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-18 07:53:12,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-18 07:53:12,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-18 07:53:12,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-18 07:53:12,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-18 07:53:12,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-18 07:53:12,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-18 07:53:12,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-18 07:53:12,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-18 07:53:12,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-18 07:53:12,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-18 07:53:12,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-18 07:53:12,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-18 07:53:12,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-18 07:53:12,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-18 07:53:12,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-18 07:53:12,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-18 07:53:12,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-18 07:53:12,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-18 07:53:12,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-18 07:53:12,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-18 07:53:12,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-18 07:53:12,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-18 07:53:12,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-18 07:53:12,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-18 07:53:12,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-18 07:53:12,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-18 07:53:12,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-18 07:53:12,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-18 07:53:12,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-18 07:53:12,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-18 07:53:12,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-18 07:53:12,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-18 07:53:12,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-18 07:53:12,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-18 07:53:12,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-18 07:53:12,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-18 07:53:12,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-18 07:53:12,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-18 07:53:12,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-18 07:53:12,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-18 07:53:12,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-18 07:53:12,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-18 07:53:12,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-18 07:53:12,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-18 07:53:12,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-18 07:53:12,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-18 07:53:12,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-18 07:53:12,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-18 07:53:12,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-18 07:53:12,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-18 07:53:12,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-18 07:53:12,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-18 07:53:12,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-18 07:53:12,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-18 07:53:12,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-18 07:53:12,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-18 07:53:12,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-18 07:53:12,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-18 07:53:12,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-18 07:53:12,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-18 07:53:12,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-18 07:53:12,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-18 07:53:12,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-18 07:53:12,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-18 07:53:12,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-18 07:53:12,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-18 07:53:12,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-18 07:53:12,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-18 07:53:12,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-18 07:53:12,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-18 07:53:12,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-18 07:53:12,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-18 07:53:12,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-18 07:53:12,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-18 07:53:12,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-18 07:53:12,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-18 07:53:12,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-18 07:53:12,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-18 07:53:12,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-18 07:53:12,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-18 07:53:12,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-18 07:53:12,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-18 07:53:12,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-18 07:53:12,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-18 07:53:12,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-18 07:53:12,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-18 07:53:12,285 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-18 07:53:12,285 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-18 07:53:12,286 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-18 07:53:13,165 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 1070 ms to try 5000 different iterations.  Found a solution that moves 4737 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1742542205842089. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039752946999079324, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.056367799708659114, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5263333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039752946999079324, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1695, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23525, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-18 07:53:13,209 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv653365683:80 , srv1393368018:82 , srv211529347:82 , srv543683809:82 , srv653383198:82 , srv1042408835:83 , srv160456350:83 , srv1753751030:83 , srv1949043756:83 , srv2127720055:83 , srv319101941:83 , srv558618581:83 , srv964686832:83 , srv1505992907:84 , srv1556642375:84 , srv1964425110:84 , srv2079952052:84 , srv2102197120:84 , srv210886203:84 , srv344646834:84 , srv668072159:84 , srv813531251:84 , srv91851259:84 , srv1026722663:85 , srv112507680:85 , srv1263920127:85 , srv1363664170:85 , srv1723157668:85 , srv2007910844:85 , srv2026875976:85 , srv2140697095:85 , srv308191181:85 , srv337674047:85 , srv421726031:85 , srv481685376:85 , srv62269457:85 , srv652413458:85 , srv704555886:85 , srv896852993:85 , srv1002936450:86 , srv104160899:86 , srv1192972907:86 , srv1347387609:86 , srv1419437673:86 , srv1514604712:86 , srv1515621783:86 , srv1515645687:86 , srv1519024577:86 , srv1533169314:86 , srv1692716423:86 , srv2070956033:86 , srv2100108773:86 , srv24078967:86 , srv298866798:86 , srv51299728:86 , srv624909926:86 , srv866842406:86 , srv995813261:86 , srv1348242334:87 , srv1450649307:87 , srv1467231617:87 , srv1655294863:87 , srv1781720329:87 , srv195514735:87 , srv304711778:87 , srv413290954:87 , srv59984462:87 , srv831312246:87 , srv834842376:87 , srv1045914756:88 , srv1331808098:88 , srv1364823596:88 , srv1484667056:88 , srv1487613277:88 , srv1504990062:88 , srv1655772079:88 , srv1709239806:88 , srv1806257125:88 , srv217006633:88 , srv462554789:88 , srv629492933:88 , srv1290934995:89 , srv1364997080:89 , srv1441765435:89 , srv1657210050:89 , srv169533209:89 , srv1976474890:89 , srv796673089:89 , srv1099009312:90 , srv1404540767:90 , srv1563463730:90 , srv1773315984:90 , srv458237854:90 , srv881899638:90 , srv1389836630:92 , srv2139413132:93 , srv1442102759:96 , srv811250498:96 , srv1244466770:98 , srv1103296073:443 }
2023-09-18 07:53:13,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-18 07:53:13,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2127720055=61, srv1773315984=46, srv834842376=93, srv51299728=77, srv24078967=65, srv704555886=88, srv413290954=72, srv1331808098=12, srv1709239806=43, srv481685376=76, srv458237854=74, srv1533169314=34, srv813531251=91, srv1519024577=33, srv896852993=96, srv1419437673=21, srv2070956033=55, srv211529347=60, srv1450649307=24, srv421726031=73, srv831312246=92, srv1487613277=27, srv62269457=81, srv1504990062=28, srv1364823596=16, srv1045914756=4, srv629492933=83, srv1404540767=20, srv2079952052=56, srv59984462=80, srv1389836630=18, srv1556642375=35, srv304711778=67, srv344646834=71, srv1042408835=3, srv1964425110=51, srv169533209=42, srv1781720329=47, srv195514735=50, srv1192972907=8, srv1655772079=39, srv1563463730=36, srv319101941=69, srv1099009312=5, srv2102197120=58, srv210886203=59, srv298866798=66, srv1753751030=45, srv1002936450=0, srv624909926=82, srv462554789=75, srv160456350=37, srv2026875976=54, srv1026722663=1, srv112507680=7, srv337674047=70, srv1723157668=44, srv1515621783=31, srv653383198=86, srv1514604712=30, srv881899638=95, srv1363664170=15, srv964686832=98, srv2007910844=53, srv1364997080=17, srv1467231617=25, srv2100108773=57, srv1441765435=22, srv1949043756=49, srv1244466770=9, srv104160899=2, srv1692716423=41, srv866842406=94, srv1263920127=10, srv1655294863=38, srv1505992907=29, srv652413458=84, srv1442102759=23, srv1484667056=26, srv1657210050=40, srv1806257125=48, srv91851259=97, srv811250498=90, srv1347387609=13, srv1393368018=19, srv1103296073=6, srv668072159=87, srv543683809=78, srv558618581=79, srv1976474890=52, srv1515645687=32, srv308191181=68, srv1290934995=11, srv2139413132=62, srv1348242334=14, srv217006633=64, srv995813261=99, srv2140697095=63, srv796673089=89, srv653365683=85} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-18 07:53:13,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-18 07:53:13,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-18 07:53:13,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-18 07:53:13,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-18 07:53:13,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-18 07:53:13,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-18 07:53:13,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-18 07:53:13,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-18 07:53:13,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-18 07:53:13,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-18 07:53:13,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-18 07:53:13,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-18 07:53:13,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-18 07:53:13,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-18 07:53:13,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-18 07:53:13,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-18 07:53:13,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-18 07:53:13,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-18 07:53:13,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-18 07:53:13,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-18 07:53:13,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-18 07:53:13,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-18 07:53:13,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-18 07:53:13,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-18 07:53:13,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-18 07:53:13,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-18 07:53:13,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-18 07:53:13,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-18 07:53:13,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-18 07:53:13,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-18 07:53:13,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-18 07:53:13,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-18 07:53:13,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-18 07:53:13,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-18 07:53:13,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-18 07:53:13,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-18 07:53:13,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-18 07:53:13,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-18 07:53:13,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-18 07:53:13,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-18 07:53:13,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-18 07:53:13,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-18 07:53:13,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-18 07:53:13,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-18 07:53:13,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-18 07:53:13,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-18 07:53:13,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-18 07:53:13,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-18 07:53:13,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-18 07:53:13,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-18 07:53:13,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-18 07:53:13,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-18 07:53:13,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-18 07:53:13,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-18 07:53:13,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-18 07:53:13,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-18 07:53:13,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-18 07:53:13,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-18 07:53:13,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-18 07:53:13,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-18 07:53:13,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-18 07:53:13,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-18 07:53:13,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-18 07:53:13,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-18 07:53:13,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-18 07:53:13,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-18 07:53:13,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-18 07:53:13,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-18 07:53:13,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-18 07:53:13,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-18 07:53:13,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-18 07:53:13,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-18 07:53:13,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-18 07:53:13,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-18 07:53:13,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-18 07:53:13,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-18 07:53:13,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-18 07:53:13,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-18 07:53:13,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-18 07:53:13,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-18 07:53:13,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-18 07:53:13,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-18 07:53:13,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-18 07:53:13,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-18 07:53:13,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-18 07:53:13,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-18 07:53:13,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-18 07:53:13,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-18 07:53:13,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-18 07:53:13,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-18 07:53:13,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-18 07:53:13,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-18 07:53:13,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-18 07:53:13,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-18 07:53:13,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-18 07:53:13,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-18 07:53:13,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-18 07:53:13,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-18 07:53:13,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-18 07:53:13,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-18 07:53:13,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-18 07:53:13,457 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039752946999079324, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.056367799708659114, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039752946999079324, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1695, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23525, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-18 07:53:13,458 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-18 07:53:13,458 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.1742210514012697, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039752946999079324, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.056367799708659114, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039752946999079324, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1695, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23525, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-18 07:53:13,854 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 562 ms to try 5000 different iterations.  Found a solution that moves 2588 regions; Going from a computed imbalance of 0.1742210514012697 to a new imbalance of 0.01685994562496296. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022381026293433793); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.034197551122936216, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.28755555555555556, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022381026293433793); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.045, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-18 07:53:13,867 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1244466770:73 , srv1442102759:77 , srv2139413132:77 , srv811250498:78 , srv1389836630:80 , srv1441765435:81 , srv1773315984:81 , srv1404540767:82 , srv1655772079:82 , srv1045914756:83 , srv1099009312:83 , srv1290934995:83 , srv1331808098:83 , srv1364997080:83 , srv1487613277:83 , srv169533209:83 , srv1709239806:83 , srv629492933:83 , srv881899638:83 , srv1347387609:84 , srv1364823596:84 , srv1419437673:84 , srv1467231617:84 , srv1484667056:84 , srv1563463730:84 , srv1655294863:84 , srv1657210050:84 , srv304711778:84 , srv458237854:84 , srv462554789:84 , srv59984462:84 , srv796673089:84 , srv1192972907:85 , srv1348242334:85 , srv1450649307:85 , srv1692716423:85 , srv1781720329:85 , srv1806257125:85 , srv1976474890:85 , srv2007910844:85 , srv217006633:85 , srv308191181:85 , srv413290954:85 , srv831312246:85 , srv866842406:85 , srv1002936450:86 , srv1026722663:86 , srv1505992907:86 , srv1533169314:86 , srv1723157668:86 , srv195514735:86 , srv2026875976:86 , srv2100108773:86 , srv24078967:86 , srv298866798:86 , srv51299728:86 , srv624909926:86 , srv668072159:86 , srv704555886:86 , srv834842376:86 , srv1042408835:87 , srv112507680:87 , srv1263920127:87 , srv1363664170:87 , srv1504990062:87 , srv1514604712:87 , srv1515621783:87 , srv1519024577:87 , srv160456350:87 , srv210886203:87 , srv2127720055:87 , srv2140697095:87 , srv337674047:87 , srv421726031:87 , srv481685376:87 , srv62269457:87 , srv896852993:87 , srv104160899:88 , srv1556642375:88 , srv1964425110:88 , srv2070956033:88 , srv2102197120:88 , srv319101941:88 , srv344646834:88 , srv995813261:88 , srv1515645687:89 , srv211529347:89 , srv543683809:89 , srv652413458:89 , srv653383198:89 , srv1393368018:90 , srv2079952052:90 , srv558618581:90 , srv653365683:90 , srv813531251:90 , srv1753751030:91 , srv1949043756:91 , srv91851259:91 , srv964686832:91 , srv1103296073:530 }
2023-09-18 07:53:13,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-18 07:53:13,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2127720055=61, srv1773315984=46, srv834842376=93, srv51299728=77, srv24078967=65, srv704555886=88, srv413290954=72, srv1331808098=12, srv1709239806=43, srv481685376=76, srv458237854=74, srv1533169314=34, srv813531251=91, srv1519024577=33, srv896852993=96, srv1419437673=21, srv2070956033=55, srv211529347=60, srv1450649307=24, srv421726031=73, srv831312246=92, srv1487613277=27, srv62269457=81, srv1504990062=28, srv1364823596=16, srv1045914756=4, srv629492933=83, srv1404540767=20, srv2079952052=56, srv59984462=80, srv1389836630=18, srv1556642375=35, srv304711778=67, srv344646834=71, srv1042408835=3, srv1964425110=51, srv169533209=42, srv1781720329=47, srv195514735=50, srv1192972907=8, srv1655772079=39, srv1563463730=36, srv319101941=69, srv1099009312=5, srv2102197120=58, srv210886203=59, srv298866798=66, srv1753751030=45, srv1002936450=0, srv624909926=82, srv462554789=75, srv160456350=37, srv2026875976=54, srv1026722663=1, srv112507680=7, srv337674047=70, srv1723157668=44, srv1515621783=31, srv653383198=86, srv1514604712=30, srv881899638=95, srv1363664170=15, srv964686832=98, srv2007910844=53, srv1364997080=17, srv1467231617=25, srv2100108773=57, srv1441765435=22, srv1949043756=49, srv1244466770=9, srv104160899=2, srv1692716423=41, srv866842406=94, srv1263920127=10, srv1655294863=38, srv1505992907=29, srv652413458=84, srv1442102759=23, srv1484667056=26, srv1657210050=40, srv1806257125=48, srv91851259=97, srv811250498=90, srv1347387609=13, srv1393368018=19, srv1103296073=6, srv668072159=87, srv543683809=78, srv558618581=79, srv1976474890=52, srv1515645687=32, srv308191181=68, srv1290934995=11, srv2139413132=62, srv1348242334=14, srv217006633=64, srv995813261=99, srv2140697095=63, srv796673089=89, srv653365683=85} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-18 07:53:13,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-18 07:53:13,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-18 07:53:13,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-18 07:53:13,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-18 07:53:13,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-18 07:53:13,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-18 07:53:13,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-18 07:53:13,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-18 07:53:13,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-18 07:53:13,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-18 07:53:13,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-18 07:53:13,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-18 07:53:13,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-18 07:53:13,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-18 07:53:13,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-18 07:53:13,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-18 07:53:13,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-18 07:53:13,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-18 07:53:13,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-18 07:53:13,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-18 07:53:13,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-18 07:53:13,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-18 07:53:13,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-18 07:53:13,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-18 07:53:13,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-18 07:53:13,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-18 07:53:13,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-18 07:53:13,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-18 07:53:13,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-18 07:53:13,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-18 07:53:13,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-18 07:53:13,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-18 07:53:13,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-18 07:53:13,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-18 07:53:13,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-18 07:53:13,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-18 07:53:13,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-18 07:53:13,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-18 07:53:13,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-18 07:53:13,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-18 07:53:13,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-18 07:53:13,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-18 07:53:13,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-18 07:53:13,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-18 07:53:13,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-18 07:53:13,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-18 07:53:13,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-18 07:53:13,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-18 07:53:13,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-18 07:53:13,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-18 07:53:13,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-18 07:53:13,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-18 07:53:13,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-18 07:53:13,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-18 07:53:13,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-18 07:53:13,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-18 07:53:13,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-18 07:53:13,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-18 07:53:13,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-18 07:53:13,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-18 07:53:13,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-18 07:53:13,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-18 07:53:13,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-18 07:53:13,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-18 07:53:13,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-18 07:53:13,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-18 07:53:13,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-18 07:53:13,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-18 07:53:13,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-18 07:53:13,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-18 07:53:13,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-18 07:53:13,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-18 07:53:13,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-18 07:53:13,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-18 07:53:13,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-18 07:53:13,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-18 07:53:13,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-18 07:53:13,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-18 07:53:13,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-18 07:53:13,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-18 07:53:13,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-18 07:53:13,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-18 07:53:13,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-18 07:53:13,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-18 07:53:13,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-18 07:53:13,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-18 07:53:13,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-18 07:53:13,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-18 07:53:13,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-18 07:53:13,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-18 07:53:13,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-18 07:53:13,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-18 07:53:13,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-18 07:53:13,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-18 07:53:13,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-18 07:53:13,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-18 07:53:13,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-18 07:53:13,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-18 07:53:13,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-18 07:53:13,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-18 07:53:13,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-18 07:53:14,124 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022381026293433793); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.034197551122936216, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022381026293433793); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.045, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-18 07:53:14,125 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-18 07:53:14,125 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.016841824061643022, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022381026293433793); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.034197551122936216, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022381026293433793); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.045, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-18 07:53:14,401 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 417 ms to try 5000 different iterations.  Found a solution that moves 1006 regions; Going from a computed imbalance of 0.016841824061643022 to a new imbalance of 6.849604560128108E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006683480480676067); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.010117158808524375); MoveCostFunction : (multiplier=7.0, imbalance=0.11177777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006683480480676067); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004166666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-18 07:53:14,403 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv91851259:80 , srv1515645687:81 , srv1504990062:82 , srv1753751030:82 , srv104160899:83 , srv1949043756:83 , srv2079952052:83 , srv652413458:83 , srv811250498:83 , srv1514604712:84 , srv1519024577:84 , srv1563463730:84 , srv1657210050:84 , srv1806257125:84 , srv195514735:84 , srv1976474890:84 , srv2070956033:84 , srv2102197120:84 , srv421726031:84 , srv458237854:84 , srv813531251:84 , srv834842376:84 , srv881899638:84 , srv964686832:84 , srv995813261:84 , srv1026722663:85 , srv1099009312:85 , srv1263920127:85 , srv1331808098:85 , srv1348242334:85 , srv1389836630:85 , srv1393368018:85 , srv1404540767:85 , srv1442102759:85 , srv1450649307:85 , srv1484667056:85 , srv1515621783:85 , srv1692716423:85 , srv1773315984:85 , srv1781720329:85 , srv217006633:85 , srv337674047:85 , srv413290954:85 , srv462554789:85 , srv51299728:85 , srv558618581:85 , srv62269457:85 , srv831312246:85 , srv866842406:85 , srv896852993:85 , srv1045914756:86 , srv1192972907:86 , srv1290934995:86 , srv1364823596:86 , srv1364997080:86 , srv1419437673:86 , srv1487613277:86 , srv1533169314:86 , srv1556642375:86 , srv1655294863:86 , srv169533209:86 , srv1964425110:86 , srv2026875976:86 , srv2100108773:86 , srv2140697095:86 , srv24078967:86 , srv298866798:86 , srv319101941:86 , srv481685376:86 , srv59984462:86 , srv624909926:86 , srv796673089:86 , srv1002936450:87 , srv1042408835:87 , srv112507680:87 , srv1347387609:87 , srv1363664170:87 , srv1467231617:87 , srv1655772079:87 , srv1723157668:87 , srv2007910844:87 , srv210886203:87 , srv211529347:87 , srv2127720055:87 , srv2139413132:87 , srv304711778:87 , srv308191181:87 , srv344646834:87 , srv543683809:87 , srv629492933:87 , srv653365683:87 , srv653383198:87 , srv668072159:87 , srv704555886:87 , srv1244466770:88 , srv1505992907:88 , srv160456350:88 , srv1709239806:88 , srv1441765435:89 , srv1103296073:544 }
2023-09-18 07:53:14,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-18 07:53:14,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2127720055=61, srv1773315984=46, srv834842376=93, srv51299728=77, srv24078967=65, srv704555886=88, srv413290954=72, srv1331808098=12, srv1709239806=43, srv481685376=76, srv458237854=74, srv1533169314=34, srv813531251=91, srv1519024577=33, srv896852993=96, srv1419437673=21, srv2070956033=55, srv211529347=60, srv1450649307=24, srv421726031=73, srv831312246=92, srv1487613277=27, srv62269457=81, srv1504990062=28, srv1364823596=16, srv1045914756=4, srv629492933=83, srv1404540767=20, srv2079952052=56, srv59984462=80, srv1389836630=18, srv1556642375=35, srv304711778=67, srv344646834=71, srv1042408835=3, srv1964425110=51, srv169533209=42, srv1781720329=47, srv195514735=50, srv1192972907=8, srv1655772079=39, srv1563463730=36, srv319101941=69, srv1099009312=5, srv2102197120=58, srv210886203=59, srv298866798=66, srv1753751030=45, srv1002936450=0, srv624909926=82, srv462554789=75, srv160456350=37, srv2026875976=54, srv1026722663=1, srv112507680=7, srv337674047=70, srv1723157668=44, srv1515621783=31, srv653383198=86, srv1514604712=30, srv881899638=95, srv1363664170=15, srv964686832=98, srv2007910844=53, srv1364997080=17, srv1467231617=25, srv2100108773=57, srv1441765435=22, srv1949043756=49, srv1244466770=9, srv104160899=2, srv1692716423=41, srv866842406=94, srv1263920127=10, srv1655294863=38, srv1505992907=29, srv652413458=84, srv1442102759=23, srv1484667056=26, srv1657210050=40, srv1806257125=48, srv91851259=97, srv811250498=90, srv1347387609=13, srv1393368018=19, srv1103296073=6, srv668072159=87, srv543683809=78, srv558618581=79, srv1976474890=52, srv1515645687=32, srv308191181=68, srv1290934995=11, srv2139413132=62, srv1348242334=14, srv217006633=64, srv995813261=99, srv2140697095=63, srv796673089=89, srv653365683=85} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-18 07:53:14,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-18 07:53:14,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-18 07:53:14,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-18 07:53:14,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-18 07:53:14,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-18 07:53:14,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-18 07:53:14,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-18 07:53:14,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-18 07:53:14,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-18 07:53:14,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-18 07:53:14,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-18 07:53:14,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-18 07:53:14,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-18 07:53:14,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-18 07:53:14,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-18 07:53:14,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-18 07:53:14,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-18 07:53:14,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-18 07:53:14,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-18 07:53:14,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-18 07:53:14,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-18 07:53:14,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-18 07:53:14,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-18 07:53:14,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-18 07:53:14,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-18 07:53:14,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-18 07:53:14,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-18 07:53:14,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-18 07:53:14,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-18 07:53:14,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-18 07:53:14,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-18 07:53:14,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-18 07:53:14,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-18 07:53:14,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-18 07:53:14,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-18 07:53:14,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-18 07:53:14,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-18 07:53:14,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-18 07:53:14,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-18 07:53:14,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-18 07:53:14,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-18 07:53:14,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-18 07:53:14,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-18 07:53:14,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-18 07:53:14,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-18 07:53:14,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-18 07:53:14,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-18 07:53:14,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-18 07:53:14,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-18 07:53:14,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-18 07:53:14,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-18 07:53:14,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-18 07:53:14,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-18 07:53:14,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-18 07:53:14,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-18 07:53:14,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-18 07:53:14,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-18 07:53:14,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-18 07:53:14,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-18 07:53:14,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-18 07:53:14,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-18 07:53:14,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-18 07:53:14,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-18 07:53:14,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-18 07:53:14,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-18 07:53:14,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-18 07:53:14,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-18 07:53:14,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-18 07:53:14,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-18 07:53:14,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-18 07:53:14,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-18 07:53:14,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-18 07:53:14,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-18 07:53:14,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-18 07:53:14,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-18 07:53:14,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-18 07:53:14,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-18 07:53:14,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-18 07:53:14,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-18 07:53:14,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-18 07:53:14,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-18 07:53:14,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-18 07:53:14,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-18 07:53:14,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-18 07:53:14,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-18 07:53:14,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-18 07:53:14,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-18 07:53:14,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-18 07:53:14,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-18 07:53:14,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-18 07:53:14,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-18 07:53:14,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-18 07:53:14,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-18 07:53:14,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-18 07:53:14,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-18 07:53:14,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-18 07:53:14,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-18 07:53:14,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-18 07:53:14,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-18 07:53:14,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-18 07:53:14,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-18 07:53:14,532 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006683480480676067); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.010117158808524375); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006683480480676067); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004166666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-18 07:53:14,532 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-18 07:53:14,532 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=6.779162934549056E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006683480480676067); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.010117158808524375); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006683480480676067); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004166666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-18 07:53:14,890 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 456 ms to try 5000 different iterations.  Found a solution that moves 297 regions; Going from a computed imbalance of 6.779162934549056E-4 to a new imbalance of 3.233620960104307E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.470732932414788E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0028029165506084056); MoveCostFunction : (multiplier=7.0, imbalance=0.033, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.470732932414788E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-18 07:53:14,891 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1002936450:83 , srv1244466770:83 , srv1363664170:83 , srv1441765435:83 , srv1533169314:83 , srv1709239806:83 , srv344646834:83 , srv458237854:83 , srv543683809:83 , srv704555886:83 , srv796673089:83 , srv964686832:83 , srv1099009312:84 , srv112507680:84 , srv1290934995:84 , srv1364997080:84 , srv1442102759:84 , srv1450649307:84 , srv1467231617:84 , srv1505992907:84 , srv1514604712:84 , srv1556642375:84 , srv1563463730:84 , srv160456350:84 , srv1655294863:84 , srv169533209:84 , srv1723157668:84 , srv1949043756:84 , srv1964425110:84 , srv1976474890:84 , srv2026875976:84 , srv2070956033:84 , srv2100108773:84 , srv210886203:84 , srv211529347:84 , srv2140697095:84 , srv24078967:84 , srv298866798:84 , srv304711778:84 , srv481685376:84 , srv558618581:84 , srv624909926:84 , srv629492933:84 , srv653383198:84 , srv1026722663:85 , srv104160899:85 , srv1042408835:85 , srv1045914756:85 , srv1192972907:85 , srv1263920127:85 , srv1347387609:85 , srv1348242334:85 , srv1364823596:85 , srv1389836630:85 , srv1393368018:85 , srv1419437673:85 , srv1484667056:85 , srv1504990062:85 , srv1515621783:85 , srv1519024577:85 , srv1657210050:85 , srv1781720329:85 , srv195514735:85 , srv2007910844:85 , srv2079952052:85 , srv2127720055:85 , srv2139413132:85 , srv217006633:85 , srv308191181:85 , srv319101941:85 , srv337674047:85 , srv413290954:85 , srv51299728:85 , srv59984462:85 , srv62269457:85 , srv652413458:85 , srv653365683:85 , srv668072159:85 , srv811250498:85 , srv813531251:85 , srv831312246:85 , srv834842376:85 , srv881899638:85 , srv896852993:85 , srv995813261:85 , srv1331808098:86 , srv1404540767:86 , srv1487613277:86 , srv1515645687:86 , srv1655772079:86 , srv1692716423:86 , srv1753751030:86 , srv1773315984:86 , srv1806257125:86 , srv2102197120:86 , srv421726031:86 , srv462554789:86 , srv866842406:86 , srv91851259:88 , srv1103296073:625 }
2023-09-18 07:53:14,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-18 07:53:14,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2127720055=61, srv1773315984=46, srv834842376=93, srv51299728=77, srv24078967=65, srv704555886=88, srv413290954=72, srv1331808098=12, srv1709239806=43, srv481685376=76, srv458237854=74, srv1533169314=34, srv813531251=91, srv1519024577=33, srv896852993=96, srv1419437673=21, srv2070956033=55, srv211529347=60, srv1450649307=24, srv421726031=73, srv831312246=92, srv1487613277=27, srv62269457=81, srv1504990062=28, srv1364823596=16, srv1045914756=4, srv629492933=83, srv1404540767=20, srv2079952052=56, srv59984462=80, srv1389836630=18, srv1556642375=35, srv304711778=67, srv344646834=71, srv1042408835=3, srv1964425110=51, srv169533209=42, srv1781720329=47, srv195514735=50, srv1192972907=8, srv1655772079=39, srv1563463730=36, srv319101941=69, srv1099009312=5, srv2102197120=58, srv210886203=59, srv298866798=66, srv1753751030=45, srv1002936450=0, srv624909926=82, srv462554789=75, srv160456350=37, srv2026875976=54, srv1026722663=1, srv112507680=7, srv337674047=70, srv1723157668=44, srv1515621783=31, srv653383198=86, srv1514604712=30, srv881899638=95, srv1363664170=15, srv964686832=98, srv2007910844=53, srv1364997080=17, srv1467231617=25, srv2100108773=57, srv1441765435=22, srv1949043756=49, srv1244466770=9, srv104160899=2, srv1692716423=41, srv866842406=94, srv1263920127=10, srv1655294863=38, srv1505992907=29, srv652413458=84, srv1442102759=23, srv1484667056=26, srv1657210050=40, srv1806257125=48, srv91851259=97, srv811250498=90, srv1347387609=13, srv1393368018=19, srv1103296073=6, srv668072159=87, srv543683809=78, srv558618581=79, srv1976474890=52, srv1515645687=32, srv308191181=68, srv1290934995=11, srv2139413132=62, srv1348242334=14, srv217006633=64, srv995813261=99, srv2140697095=63, srv796673089=89, srv653365683=85} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-18 07:53:14,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-18 07:53:14,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-18 07:53:14,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-18 07:53:14,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-18 07:53:14,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-18 07:53:14,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-18 07:53:14,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-18 07:53:14,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-18 07:53:14,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-18 07:53:14,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-18 07:53:14,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-18 07:53:14,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-18 07:53:14,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-18 07:53:14,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-18 07:53:14,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-18 07:53:14,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-18 07:53:14,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-18 07:53:14,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-18 07:53:14,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-18 07:53:14,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-18 07:53:14,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-18 07:53:14,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-18 07:53:14,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-18 07:53:14,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-18 07:53:14,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-18 07:53:14,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-18 07:53:14,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-18 07:53:14,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-18 07:53:14,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-18 07:53:14,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-18 07:53:14,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-18 07:53:14,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-18 07:53:14,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-18 07:53:14,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-18 07:53:14,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-18 07:53:14,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-18 07:53:14,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-18 07:53:14,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-18 07:53:14,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-18 07:53:14,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-18 07:53:14,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-18 07:53:14,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-18 07:53:14,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-18 07:53:14,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-18 07:53:14,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-18 07:53:14,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-18 07:53:14,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-18 07:53:14,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-18 07:53:14,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-18 07:53:14,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-18 07:53:14,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-18 07:53:14,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-18 07:53:14,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-18 07:53:14,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-18 07:53:14,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-18 07:53:14,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-18 07:53:14,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-18 07:53:14,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-18 07:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-18 07:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-18 07:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-18 07:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-18 07:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-18 07:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-18 07:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-18 07:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-18 07:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-18 07:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-18 07:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-18 07:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-18 07:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-18 07:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-18 07:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-18 07:53:14,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-18 07:53:14,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-18 07:53:14,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-18 07:53:14,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-18 07:53:14,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-18 07:53:14,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-18 07:53:14,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-18 07:53:14,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-18 07:53:14,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-18 07:53:14,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-18 07:53:14,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-18 07:53:14,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-18 07:53:14,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-18 07:53:14,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-18 07:53:14,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-18 07:53:14,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-18 07:53:14,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-18 07:53:14,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-18 07:53:14,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-18 07:53:14,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-18 07:53:14,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-18 07:53:14,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-18 07:53:14,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-18 07:53:14,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-18 07:53:14,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-18 07:53:14,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-18 07:53:14,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-18 07:53:14,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-18 07:53:15,015 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.470732932414788E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0028029165506084056); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.470732932414788E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-18 07:53:15,015 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-18 07:53:15,015 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=3.0256571152039224E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.470732932414788E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0028029165506084056); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.470732932414788E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-18 07:53:15,254 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 333 ms to try 5000 different iterations.  Found a solution that moves 41 regions; Going from a computed imbalance of 3.0256571152039224E-5 to a new imbalance of 9.335206245650324E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.004555555555555556); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-18 07:53:15,254 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv112507680:83 , srv1364823596:83 , srv1404540767:83 , srv1487613277:83 , srv1515621783:83 , srv1655772079:83 , srv1806257125:83 , srv217006633:83 , srv462554789:83 , srv813531251:83 , srv91851259:83 , srv995813261:83 , srv1002936450:84 , srv104160899:84 , srv1042408835:84 , srv1045914756:84 , srv1099009312:84 , srv1192972907:84 , srv1244466770:84 , srv1263920127:84 , srv1290934995:84 , srv1331808098:84 , srv1347387609:84 , srv1348242334:84 , srv1363664170:84 , srv1364997080:84 , srv1389836630:84 , srv1393368018:84 , srv1441765435:84 , srv1442102759:84 , srv1467231617:84 , srv1484667056:84 , srv1504990062:84 , srv1505992907:84 , srv1515645687:84 , srv1519024577:84 , srv1556642375:84 , srv1563463730:84 , srv160456350:84 , srv1657210050:84 , srv1692716423:84 , srv169533209:84 , srv1709239806:84 , srv1723157668:84 , srv1753751030:84 , srv1773315984:84 , srv1781720329:84 , srv195514735:84 , srv1964425110:84 , srv1976474890:84 , srv2007910844:84 , srv2070956033:84 , srv2079952052:84 , srv2100108773:84 , srv2102197120:84 , srv210886203:84 , srv211529347:84 , srv2127720055:84 , srv2139413132:84 , srv2140697095:84 , srv24078967:84 , srv298866798:84 , srv304711778:84 , srv308191181:84 , srv319101941:84 , srv337674047:84 , srv344646834:84 , srv413290954:84 , srv421726031:84 , srv481685376:84 , srv51299728:84 , srv558618581:84 , srv59984462:84 , srv62269457:84 , srv624909926:84 , srv629492933:84 , srv652413458:84 , srv653365683:84 , srv653383198:84 , srv668072159:84 , srv796673089:84 , srv811250498:84 , srv831312246:84 , srv834842376:84 , srv866842406:84 , srv881899638:84 , srv896852993:84 , srv1026722663:85 , srv1419437673:85 , srv1450649307:85 , srv1514604712:85 , srv1533169314:85 , srv1655294863:85 , srv1949043756:85 , srv2026875976:85 , srv458237854:85 , srv543683809:85 , srv704555886:85 , srv964686832:85 , srv1103296073:684 }
2023-09-18 07:53:15,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-18 07:53:15,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2127720055=61, srv1773315984=46, srv834842376=93, srv51299728=77, srv24078967=65, srv704555886=88, srv413290954=72, srv1331808098=12, srv1709239806=43, srv481685376=76, srv458237854=74, srv1533169314=34, srv813531251=91, srv1519024577=33, srv896852993=96, srv1419437673=21, srv2070956033=55, srv211529347=60, srv1450649307=24, srv421726031=73, srv831312246=92, srv1487613277=27, srv62269457=81, srv1504990062=28, srv1364823596=16, srv1045914756=4, srv629492933=83, srv1404540767=20, srv2079952052=56, srv59984462=80, srv1389836630=18, srv1556642375=35, srv304711778=67, srv344646834=71, srv1042408835=3, srv1964425110=51, srv169533209=42, srv1781720329=47, srv195514735=50, srv1192972907=8, srv1655772079=39, srv1563463730=36, srv319101941=69, srv1099009312=5, srv2102197120=58, srv210886203=59, srv298866798=66, srv1753751030=45, srv1002936450=0, srv624909926=82, srv462554789=75, srv160456350=37, srv2026875976=54, srv1026722663=1, srv112507680=7, srv337674047=70, srv1723157668=44, srv1515621783=31, srv653383198=86, srv1514604712=30, srv881899638=95, srv1363664170=15, srv964686832=98, srv2007910844=53, srv1364997080=17, srv1467231617=25, srv2100108773=57, srv1441765435=22, srv1949043756=49, srv1244466770=9, srv104160899=2, srv1692716423=41, srv866842406=94, srv1263920127=10, srv1655294863=38, srv1505992907=29, srv652413458=84, srv1442102759=23, srv1484667056=26, srv1657210050=40, srv1806257125=48, srv91851259=97, srv811250498=90, srv1347387609=13, srv1393368018=19, srv1103296073=6, srv668072159=87, srv543683809=78, srv558618581=79, srv1976474890=52, srv1515645687=32, srv308191181=68, srv1290934995=11, srv2139413132=62, srv1348242334=14, srv217006633=64, srv995813261=99, srv2140697095=63, srv796673089=89, srv653365683=85} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-18 07:53:15,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-18 07:53:15,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-18 07:53:15,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-18 07:53:15,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-18 07:53:15,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-18 07:53:15,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-18 07:53:15,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-18 07:53:15,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-18 07:53:15,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-18 07:53:15,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-18 07:53:15,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-18 07:53:15,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-18 07:53:15,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-18 07:53:15,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-18 07:53:15,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-18 07:53:15,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-18 07:53:15,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-18 07:53:15,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-18 07:53:15,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-18 07:53:15,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-18 07:53:15,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-18 07:53:15,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-18 07:53:15,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-18 07:53:15,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-18 07:53:15,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-18 07:53:15,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-18 07:53:15,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-18 07:53:15,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-18 07:53:15,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-18 07:53:15,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-18 07:53:15,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-18 07:53:15,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-18 07:53:15,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-18 07:53:15,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-18 07:53:15,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-18 07:53:15,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-18 07:53:15,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-18 07:53:15,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-18 07:53:15,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-18 07:53:15,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-18 07:53:15,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-18 07:53:15,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-18 07:53:15,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-18 07:53:15,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-18 07:53:15,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-18 07:53:15,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-18 07:53:15,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-18 07:53:15,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-18 07:53:15,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-18 07:53:15,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-18 07:53:15,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-18 07:53:15,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-18 07:53:15,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-18 07:53:15,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-18 07:53:15,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-18 07:53:15,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-18 07:53:15,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-18 07:53:15,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-18 07:53:15,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-18 07:53:15,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-18 07:53:15,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-18 07:53:15,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-18 07:53:15,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-18 07:53:15,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-18 07:53:15,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-18 07:53:15,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-18 07:53:15,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-18 07:53:15,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-18 07:53:15,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-18 07:53:15,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-18 07:53:15,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-18 07:53:15,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-18 07:53:15,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-18 07:53:15,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-18 07:53:15,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-18 07:53:15,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-18 07:53:15,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-18 07:53:15,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-18 07:53:15,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-18 07:53:15,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-18 07:53:15,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-18 07:53:15,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-18 07:53:15,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-18 07:53:15,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-18 07:53:15,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-18 07:53:15,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-18 07:53:15,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-18 07:53:15,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-18 07:53:15,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-18 07:53:15,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-18 07:53:15,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-18 07:53:15,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-18 07:53:15,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-18 07:53:15,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-18 07:53:15,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-18 07:53:15,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-18 07:53:15,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-18 07:53:15,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-18 07:53:15,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-18 07:53:15,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-18 07:53:15,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-18 07:53:15,379 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=9.048118109592553E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-18 07:53:15,379 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv112507680:83 , srv1364823596:83 , srv1404540767:83 , srv1487613277:83 , srv1515621783:83 , srv1655772079:83 , srv1806257125:83 , srv217006633:83 , srv462554789:83 , srv813531251:83 , srv91851259:83 , srv995813261:83 , srv1002936450:84 , srv104160899:84 , srv1042408835:84 , srv1045914756:84 , srv1099009312:84 , srv1192972907:84 , srv1244466770:84 , srv1263920127:84 , srv1290934995:84 , srv1331808098:84 , srv1347387609:84 , srv1348242334:84 , srv1363664170:84 , srv1364997080:84 , srv1389836630:84 , srv1393368018:84 , srv1441765435:84 , srv1442102759:84 , srv1467231617:84 , srv1484667056:84 , srv1504990062:84 , srv1505992907:84 , srv1515645687:84 , srv1519024577:84 , srv1556642375:84 , srv1563463730:84 , srv160456350:84 , srv1657210050:84 , srv1692716423:84 , srv169533209:84 , srv1709239806:84 , srv1723157668:84 , srv1753751030:84 , srv1773315984:84 , srv1781720329:84 , srv195514735:84 , srv1964425110:84 , srv1976474890:84 , srv2007910844:84 , srv2070956033:84 , srv2079952052:84 , srv2100108773:84 , srv2102197120:84 , srv210886203:84 , srv211529347:84 , srv2127720055:84 , srv2139413132:84 , srv2140697095:84 , srv24078967:84 , srv298866798:84 , srv304711778:84 , srv308191181:84 , srv319101941:84 , srv337674047:84 , srv344646834:84 , srv413290954:84 , srv421726031:84 , srv481685376:84 , srv51299728:84 , srv558618581:84 , srv59984462:84 , srv62269457:84 , srv624909926:84 , srv629492933:84 , srv652413458:84 , srv653365683:84 , srv653383198:84 , srv668072159:84 , srv796673089:84 , srv811250498:84 , srv831312246:84 , srv834842376:84 , srv866842406:84 , srv881899638:84 , srv896852993:84 , srv1026722663:85 , srv1419437673:85 , srv1450649307:85 , srv1514604712:85 , srv1533169314:85 , srv1655294863:85 , srv1949043756:85 , srv2026875976:85 , srv458237854:85 , srv543683809:85 , srv704555886:85 , srv964686832:85 , srv1103296073:684 }
2023-09-18 07:53:15,405 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=276 (was 248) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 169), AvailableMemoryMB=9381 (was 10005)
2023-09-18 07:53:15,413 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=276, ProcessCount=169, AvailableMemoryMB=9380
2023-09-18 07:53:15,414 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-09-18 07:53:15,414 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-09-18 07:53:15,423 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1053231667:84 , srv1208322606:84 , srv1219845490:84 , srv1328633432:84 , srv1384016940:84 , srv1409951874:84 , srv1446642058:84 , srv15769418:84 , srv1599383061:84 , srv1601671712:84 , srv1952849469:84 , srv1982209221:84 , srv1999745456:84 , srv2008822712:84 , srv2011200131:84 , srv216835760:84 , srv297294805:84 , srv325200022:84 , srv356734066:84 , srv557542474:84 , srv589204578:84 , srv597775904:84 , srv62931941:84 , srv69224213:84 , srv72019808:84 , srv775345624:84 , srv780363525:84 , srv788506224:84 , srv792150592:84 , srv207934450:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-09-18 07:53:15,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-18 07:53:15,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv589204578=21, srv1219845490=2, srv356734066=19, srv1446642058=6, srv1601671712=9, srv1208322606=1, srv1599383061=8, srv297294805=17, srv325200022=18, srv15769418=7, srv2008822712=13, srv1982209221=11, srv780363525=27, srv792150592=29, srv1384016940=4, srv557542474=20, srv1409951874=5, srv1952849469=10, srv207934450=15, srv1053231667=0, srv1328633432=3, srv216835760=16, srv62931941=23, srv69224213=24, srv72019808=25, srv2011200131=14, srv775345624=26, srv788506224=28, srv597775904=22, srv1999745456=12} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-18 07:53:15,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-18 07:53:15,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-18 07:53:15,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-18 07:53:15,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-18 07:53:15,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-18 07:53:15,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-18 07:53:15,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-18 07:53:15,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-18 07:53:15,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-18 07:53:15,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-18 07:53:15,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-18 07:53:15,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-18 07:53:15,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-18 07:53:15,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-18 07:53:15,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-18 07:53:15,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-18 07:53:15,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-18 07:53:15,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-18 07:53:15,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-18 07:53:15,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-18 07:53:15,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-18 07:53:15,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-18 07:53:15,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-18 07:53:15,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-18 07:53:15,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-18 07:53:15,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-18 07:53:15,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-18 07:53:15,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-18 07:53:15,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-18 07:53:15,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-18 07:53:15,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-09-18 07:53:15,450 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.2200035543733405, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-18 07:53:15,450 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912886267280332, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.2200035543733405, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-09-18 07:55:15,437 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6417774 different iterations.  Found a solution that moves 1862 regions; Going from a computed imbalance of 0.9912886267280332 to a new imbalance of 4.346000888939571E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6896296296296296, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-18 07:55:15,445 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1053231667:90 , srv1208322606:90 , srv1219845490:90 , srv1328633432:90 , srv1384016940:90 , srv1409951874:90 , srv1446642058:90 , srv15769418:90 , srv1599383061:90 , srv1601671712:90 , srv1952849469:90 , srv1982209221:90 , srv1999745456:90 , srv2008822712:90 , srv2011200131:90 , srv207934450:90 , srv216835760:90 , srv297294805:90 , srv325200022:90 , srv356734066:90 , srv557542474:90 , srv589204578:90 , srv597775904:90 , srv62931941:90 , srv69224213:90 , srv72019808:90 , srv775345624:90 , srv780363525:90 , srv788506224:90 , srv792150592:90 }
2023-09-18 07:55:15,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-18 07:55:15,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv589204578=21, srv1219845490=2, srv356734066=19, srv1446642058=6, srv1601671712=9, srv1208322606=1, srv1599383061=8, srv297294805=17, srv325200022=18, srv15769418=7, srv2008822712=13, srv1982209221=11, srv780363525=27, srv792150592=29, srv1384016940=4, srv557542474=20, srv1409951874=5, srv1952849469=10, srv207934450=15, srv1053231667=0, srv1328633432=3, srv216835760=16, srv62931941=23, srv69224213=24, srv72019808=25, srv2011200131=14, srv775345624=26, srv788506224=28, srv597775904=22, srv1999745456=12} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-18 07:55:15,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-18 07:55:15,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-18 07:55:15,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-18 07:55:15,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-18 07:55:15,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-18 07:55:15,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-18 07:55:15,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-18 07:55:15,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-18 07:55:15,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-18 07:55:15,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-18 07:55:15,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-18 07:55:15,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-18 07:55:15,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-18 07:55:15,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-18 07:55:15,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-18 07:55:15,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-18 07:55:15,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-18 07:55:15,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-18 07:55:15,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-18 07:55:15,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-18 07:55:15,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-18 07:55:15,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-18 07:55:15,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-18 07:55:15,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-18 07:55:15,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-18 07:55:15,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-18 07:55:15,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-18 07:55:15,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-18 07:55:15,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-18 07:55:15,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-18 07:55:15,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-09-18 07:55:15,467 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-18 07:55:15,467 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1053231667:90 , srv1208322606:90 , srv1219845490:90 , srv1328633432:90 , srv1384016940:90 , srv1409951874:90 , srv1446642058:90 , srv15769418:90 , srv1599383061:90 , srv1601671712:90 , srv1952849469:90 , srv1982209221:90 , srv1999745456:90 , srv2008822712:90 , srv2011200131:90 , srv207934450:90 , srv216835760:90 , srv297294805:90 , srv325200022:90 , srv356734066:90 , srv557542474:90 , srv589204578:90 , srv597775904:90 , srv62931941:90 , srv69224213:90 , srv72019808:90 , srv775345624:90 , srv780363525:90 , srv788506224:90 , srv792150592:90 }
2023-09-18 07:55:15,479 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=139 (was 276), ProcessCount=167 (was 169), AvailableMemoryMB=8576 (was 9380)