2023-12-30 05:53:18,996 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-12-30 05:53:19,313 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-12-30 05:53:19,427 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-12-30 05:53:19,492 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-12-30 05:53:19,499 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-12-30 05:53:19,515 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=258, ProcessCount=169, AvailableMemoryMB=5423
2023-12-30 05:53:19,517 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-12-30 05:53:19,517 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-12-30 05:53:19,656 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1028001532:84 , srv1051007714:84 , srv1069642509:84 , srv1072157603:84 , srv1144242971:84 , srv1171503074:84 , srv1196718335:84 , srv1198735154:84 , srv1213130142:84 , srv1227974848:84 , srv1231841844:84 , srv1232681931:84 , srv123376764:84 , srv1269244740:84 , srv1281502886:84 , srv1290915992:84 , srv1316015936:84 , srv1374238790:84 , srv1399896240:84 , srv1423488730:84 , srv147021237:84 , srv1478615615:84 , srv1498707409:84 , srv1524005881:84 , srv1527681684:84 , srv154686500:84 , srv1597422589:84 , srv159872542:84 , srv160892268:84 , srv1611700567:84 , srv1614432340:84 , srv1620181053:84 , srv1652031645:84 , srv1662497634:84 , srv1685301828:84 , srv1727381088:84 , srv1778883681:84 , srv178896665:84 , srv1798728805:84 , srv1874290241:84 , srv189938520:84 , srv1913532144:84 , srv1965103883:84 , srv1977216623:84 , srv1990529924:84 , srv199815715:84 , srv202481508:84 , srv2033217985:84 , srv2038769188:84 , srv2045412648:84 , srv2052420381:84 , srv2061011169:84 , srv2076464993:84 , srv2080464196:84 , srv2096104151:84 , srv2103801351:84 , srv2107816420:84 , srv2110350468:84 , srv218976722:84 , srv237354130:84 , srv250695334:84 , srv255990223:84 , srv257539047:84 , srv265642781:84 , srv299638338:84 , srv300066544:84 , srv304126051:84 , srv328035743:84 , srv354681993:84 , srv385150683:84 , srv397535460:84 , srv402623599:84 , srv473084183:84 , srv473736616:84 , srv511787462:84 , srv550298934:84 , srv55150560:84 , srv633533415:84 , srv645911735:84 , srv646747398:84 , srv69549737:84 , srv703172514:84 , srv752628968:84 , srv770852387:84 , srv787221743:84 , srv811608707:84 , srv815387775:84 , srv820246453:84 , srv838872599:84 , srv866077660:84 , srv884085010:84 , srv902727305:84 , srv905333270:84 , srv930130903:84 , srv935925832:84 , srv942970412:84 , srv969610978:84 , srv985722840:84 , srv998631095:84 , srv282026324:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-12-30 05:53:19,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-30 05:53:19,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1069642509=2, srv304126051=67, srv1072157603=3, srv1213130142=8, srv820246453=88, srv159872542=27, srv178896665=37, srv2038769188=48, srv511787462=75, srv1231841844=10, srv1290915992=15, srv147021237=20, srv1144242971=4, srv1597422589=26, srv1316015936=16, srv189938520=40, srv255990223=61, srv1281502886=14, srv1478615615=21, srv250695334=60, srv902727305=92, srv237354130=59, srv299638338=65, srv1232681931=11, srv328035743=68, srv787221743=85, srv385150683=70, srv218976722=58, srv1498707409=22, srv1874290241=39, srv2103801351=55, srv1977216623=43, srv2110350468=57, srv550298934=76, srv645911735=79, srv1913532144=41, srv1652031645=32, srv1051007714=1, srv1620181053=31, srv154686500=25, srv646747398=80, srv1399896240=18, srv265642781=63, srv2080464196=53, srv935925832=95, srv473084183=73, srv1269244740=13, srv1778883681=36, srv2061011169=51, srv1198735154=7, srv2045412648=49, srv884085010=91, srv282026324=64, srv2033217985=47, srv815387775=87, srv633533415=78, srv1227974848=9, srv199815715=45, srv123376764=12, srv985722840=98, srv354681993=69, srv703172514=82, srv1685301828=34, srv1965103883=42, srv942970412=96, srv905333270=93, srv838872599=89, srv2076464993=52, srv202481508=46, srv2052420381=50, srv69549737=81, srv1798728805=38, srv811608707=86, srv1662497634=33, srv402623599=72, srv1196718335=6, srv866077660=90, srv2096104151=54, srv55150560=77, srv160892268=28, srv1527681684=24, srv473736616=74, srv1028001532=0, srv300066544=66, srv1423488730=19, srv397535460=71, srv1614432340=30, srv770852387=84, srv969610978=97, srv930130903=94, srv1171503074=5, srv1727381088=35, srv2107816420=56, srv1990529924=44, srv752628968=83, srv1524005881=23, srv1611700567=29, srv257539047=62, srv998631095=99, srv1374238790=17} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-30 05:53:19,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-30 05:53:19,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-30 05:53:19,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-30 05:53:19,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-30 05:53:19,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-30 05:53:19,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-30 05:53:19,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-30 05:53:19,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-30 05:53:19,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-30 05:53:19,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-30 05:53:19,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-30 05:53:19,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-30 05:53:19,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-30 05:53:19,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-30 05:53:19,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-30 05:53:19,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-30 05:53:19,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-30 05:53:19,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-30 05:53:19,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-30 05:53:19,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-30 05:53:19,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-30 05:53:19,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-30 05:53:19,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-30 05:53:19,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-30 05:53:19,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-30 05:53:19,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-30 05:53:19,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-30 05:53:19,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-30 05:53:19,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-30 05:53:19,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-30 05:53:19,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-30 05:53:19,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-30 05:53:19,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-30 05:53:19,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-30 05:53:19,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-30 05:53:19,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-30 05:53:19,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-30 05:53:19,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-30 05:53:19,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-30 05:53:19,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-30 05:53:19,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-30 05:53:19,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-30 05:53:19,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-30 05:53:19,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-30 05:53:19,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-30 05:53:19,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-30 05:53:19,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-30 05:53:19,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-30 05:53:19,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-30 05:53:19,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-30 05:53:19,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-30 05:53:19,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-30 05:53:19,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-30 05:53:19,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-30 05:53:19,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-30 05:53:19,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-30 05:53:19,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-30 05:53:19,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-30 05:53:19,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-30 05:53:19,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-30 05:53:19,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-30 05:53:19,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-30 05:53:19,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-30 05:53:19,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-30 05:53:19,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-30 05:53:19,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-30 05:53:19,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-30 05:53:19,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-30 05:53:19,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-30 05:53:19,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-30 05:53:19,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-30 05:53:19,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-30 05:53:19,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-30 05:53:19,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-30 05:53:19,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-30 05:53:19,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-30 05:53:19,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-30 05:53:19,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-30 05:53:19,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-30 05:53:19,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-30 05:53:19,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-30 05:53:19,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-30 05:53:19,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-30 05:53:19,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-30 05:53:19,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-30 05:53:19,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-30 05:53:19,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-30 05:53:19,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-30 05:53:19,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-30 05:53:19,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-30 05:53:19,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-30 05:53:19,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-30 05:53:19,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-30 05:53:19,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-30 05:53:19,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-30 05:53:19,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-30 05:53:19,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-30 05:53:19,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-30 05:53:19,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-30 05:53:19,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-30 05:53:19,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-30 05:53:20,154 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-30 05:53:20,155 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-12-30 05:53:20,155 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-12-30 05:53:20,609 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 785 ms to try 5000 different iterations.  Found a solution that moves 4670 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17871205136717097. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04238964973068086, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05564235643289348, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5188888888888888, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04238964973068086, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17383333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24133333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-30 05:53:20,620 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1069642509:82 , srv159872542:82 , srv199815715:82 , srv2045412648:82 , srv402623599:82 , srv985722840:82 , srv998631095:82 , srv1374238790:83 , srv1399896240:83 , srv1611700567:83 , srv1662497634:83 , srv2080464196:83 , srv2110350468:83 , srv250695334:83 , srv866077660:83 , srv1227974848:84 , srv1874290241:84 , srv300066544:84 , srv354681993:84 , srv930130903:84 , srv1198735154:85 , srv1281502886:85 , srv1290915992:85 , srv1498707409:85 , srv1524005881:85 , srv1652031645:85 , srv189938520:85 , srv202481508:85 , srv2033217985:85 , srv2107816420:85 , srv218976722:85 , srv257539047:85 , srv299638338:85 , srv473736616:85 , srv633533415:85 , srv69549737:85 , srv838872599:85 , srv935925832:85 , srv969610978:85 , srv1072157603:86 , srv1231841844:86 , srv1423488730:86 , srv154686500:86 , srv1614432340:86 , srv1620181053:86 , srv178896665:86 , srv2103801351:86 , srv328035743:86 , srv397535460:86 , srv473084183:86 , srv550298934:86 , srv55150560:86 , srv884085010:86 , srv1144242971:87 , srv1196718335:87 , srv147021237:87 , srv1527681684:87 , srv1685301828:87 , srv1727381088:87 , srv1798728805:87 , srv1913532144:87 , srv2052420381:87 , srv2061011169:87 , srv2096104151:87 , srv255990223:87 , srv265642781:87 , srv304126051:87 , srv385150683:87 , srv645911735:87 , srv752628968:87 , srv787221743:87 , srv815387775:87 , srv902727305:87 , srv1213130142:88 , srv1232681931:88 , srv123376764:88 , srv1597422589:88 , srv1977216623:88 , srv1990529924:88 , srv2038769188:88 , srv237354130:88 , srv511787462:88 , srv703172514:88 , srv770852387:88 , srv811608707:88 , srv942970412:88 , srv1269244740:89 , srv646747398:89 , srv820246453:89 , srv905333270:89 , srv1171503074:90 , srv1316015936:90 , srv1478615615:90 , srv2076464993:90 , srv1028001532:91 , srv1051007714:91 , srv160892268:91 , srv1965103883:91 , srv1778883681:92 , srv282026324:467 }
2023-12-30 05:53:20,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-30 05:53:20,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1069642509=2, srv304126051=67, srv1072157603=3, srv1213130142=8, srv820246453=88, srv159872542=27, srv178896665=37, srv2038769188=48, srv511787462=75, srv1231841844=10, srv1290915992=15, srv147021237=20, srv1144242971=4, srv1597422589=26, srv1316015936=16, srv189938520=40, srv255990223=61, srv1281502886=14, srv1478615615=21, srv250695334=60, srv902727305=92, srv237354130=59, srv299638338=65, srv1232681931=11, srv328035743=68, srv787221743=85, srv385150683=70, srv218976722=58, srv1498707409=22, srv1874290241=39, srv2103801351=55, srv1977216623=43, srv2110350468=57, srv550298934=76, srv645911735=79, srv1913532144=41, srv1652031645=32, srv1051007714=1, srv1620181053=31, srv154686500=25, srv646747398=80, srv1399896240=18, srv265642781=63, srv2080464196=53, srv935925832=95, srv473084183=73, srv1269244740=13, srv1778883681=36, srv2061011169=51, srv1198735154=7, srv2045412648=49, srv884085010=91, srv282026324=64, srv2033217985=47, srv815387775=87, srv633533415=78, srv1227974848=9, srv199815715=45, srv123376764=12, srv985722840=98, srv354681993=69, srv703172514=82, srv1685301828=34, srv1965103883=42, srv942970412=96, srv905333270=93, srv838872599=89, srv2076464993=52, srv202481508=46, srv2052420381=50, srv69549737=81, srv1798728805=38, srv811608707=86, srv1662497634=33, srv402623599=72, srv1196718335=6, srv866077660=90, srv2096104151=54, srv55150560=77, srv160892268=28, srv1527681684=24, srv473736616=74, srv1028001532=0, srv300066544=66, srv1423488730=19, srv397535460=71, srv1614432340=30, srv770852387=84, srv969610978=97, srv930130903=94, srv1171503074=5, srv1727381088=35, srv2107816420=56, srv1990529924=44, srv752628968=83, srv1524005881=23, srv1611700567=29, srv257539047=62, srv998631095=99, srv1374238790=17} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-30 05:53:20,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-30 05:53:20,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-30 05:53:20,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-30 05:53:20,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-30 05:53:20,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-30 05:53:20,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-30 05:53:20,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-30 05:53:20,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-30 05:53:20,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-30 05:53:20,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-30 05:53:20,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-30 05:53:20,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-30 05:53:20,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-30 05:53:20,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-30 05:53:20,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-30 05:53:20,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-30 05:53:20,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-30 05:53:20,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-30 05:53:20,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-30 05:53:20,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-30 05:53:20,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-30 05:53:20,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-30 05:53:20,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-30 05:53:20,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-30 05:53:20,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-30 05:53:20,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-30 05:53:20,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-30 05:53:20,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-30 05:53:20,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-30 05:53:20,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-30 05:53:20,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-30 05:53:20,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-30 05:53:20,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-30 05:53:20,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-30 05:53:20,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-30 05:53:20,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-30 05:53:20,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-30 05:53:20,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-30 05:53:20,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-30 05:53:20,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-30 05:53:20,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-30 05:53:20,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-30 05:53:20,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-30 05:53:20,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-30 05:53:20,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-30 05:53:20,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-30 05:53:20,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-30 05:53:20,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-30 05:53:20,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-30 05:53:20,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-30 05:53:20,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-30 05:53:20,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-30 05:53:20,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-30 05:53:20,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-30 05:53:20,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-30 05:53:20,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-30 05:53:20,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-30 05:53:20,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-30 05:53:20,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-30 05:53:20,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-30 05:53:20,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-30 05:53:20,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-30 05:53:20,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-30 05:53:20,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-30 05:53:20,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-30 05:53:20,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-30 05:53:20,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-30 05:53:20,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-30 05:53:20,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-30 05:53:20,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-30 05:53:20,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-30 05:53:20,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-30 05:53:20,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-30 05:53:20,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-30 05:53:20,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-30 05:53:20,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-30 05:53:20,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-30 05:53:20,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-30 05:53:20,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-30 05:53:20,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-30 05:53:20,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-30 05:53:20,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-30 05:53:20,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-30 05:53:20,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-30 05:53:20,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-30 05:53:20,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-30 05:53:20,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-30 05:53:20,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-30 05:53:20,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-30 05:53:20,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-30 05:53:20,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-30 05:53:20,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-30 05:53:20,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-30 05:53:20,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-30 05:53:20,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-30 05:53:20,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-30 05:53:20,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-30 05:53:20,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-30 05:53:20,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-30 05:53:20,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-30 05:53:20,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-30 05:53:20,807 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04238964973068086, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05564235643289348, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04238964973068086, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17383333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24133333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-30 05:53:20,808 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-12-30 05:53:20,808 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17867935132825902, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04238964973068086, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05564235643289348, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04238964973068086, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17383333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24133333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-12-30 05:53:21,105 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 422 ms to try 5000 different iterations.  Found a solution that moves 2642 regions; Going from a computed imbalance of 0.17867935132825902 to a new imbalance of 0.015176097258068384. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02440921857343055); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.036203068617575, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29355555555555557, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02440921857343055); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04608333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-30 05:53:21,113 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1051007714:78 , srv1028001532:79 , srv1478615615:80 , srv1597422589:81 , srv160892268:81 , srv1778883681:81 , srv1965103883:81 , srv2076464993:81 , srv1232681931:82 , srv1269244740:82 , srv1316015936:82 , srv255990223:82 , srv385150683:82 , srv646747398:82 , srv1171503074:83 , srv1913532144:83 , srv2038769188:83 , srv2052420381:83 , srv511787462:83 , srv645911735:83 , srv1213130142:84 , srv123376764:84 , srv1727381088:84 , srv1990529924:84 , srv2096104151:84 , srv550298934:84 , srv703172514:84 , srv770852387:84 , srv884085010:84 , srv902727305:84 , srv905333270:84 , srv942970412:84 , srv1144242971:85 , srv1524005881:85 , srv154686500:85 , srv189938520:85 , srv1977216623:85 , srv2061011169:85 , srv2107816420:85 , srv299638338:85 , srv304126051:85 , srv354681993:85 , srv752628968:85 , srv787221743:85 , srv820246453:85 , srv1072157603:86 , srv1196718335:86 , srv1198735154:86 , srv1227974848:86 , srv1231841844:86 , srv147021237:86 , srv1498707409:86 , srv1527681684:86 , srv1620181053:86 , srv1685301828:86 , srv1798728805:86 , srv2033217985:86 , srv265642781:86 , srv811608707:86 , srv815387775:86 , srv838872599:86 , srv1290915992:87 , srv1611700567:87 , srv1614432340:87 , srv1652031645:87 , srv1874290241:87 , srv202481508:87 , srv2110350468:87 , srv218976722:87 , srv237354130:87 , srv473084183:87 , srv473736616:87 , srv55150560:87 , srv633533415:87 , srv935925832:87 , srv1069642509:88 , srv1281502886:88 , srv1399896240:88 , srv159872542:88 , srv178896665:88 , srv2080464196:88 , srv257539047:88 , srv328035743:88 , srv69549737:88 , srv969610978:88 , srv1423488730:89 , srv1662497634:89 , srv2045412648:89 , srv2103801351:89 , srv300066544:89 , srv397535460:89 , srv866077660:89 , srv930130903:89 , srv250695334:90 , srv985722840:90 , srv998631095:90 , srv199815715:91 , srv1374238790:92 , srv402623599:92 , srv282026324:524 }
2023-12-30 05:53:21,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-30 05:53:21,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1069642509=2, srv304126051=67, srv1072157603=3, srv1213130142=8, srv820246453=88, srv159872542=27, srv178896665=37, srv2038769188=48, srv511787462=75, srv1231841844=10, srv1290915992=15, srv147021237=20, srv1144242971=4, srv1597422589=26, srv1316015936=16, srv189938520=40, srv255990223=61, srv1281502886=14, srv1478615615=21, srv250695334=60, srv902727305=92, srv237354130=59, srv299638338=65, srv1232681931=11, srv328035743=68, srv787221743=85, srv385150683=70, srv218976722=58, srv1498707409=22, srv1874290241=39, srv2103801351=55, srv1977216623=43, srv2110350468=57, srv550298934=76, srv645911735=79, srv1913532144=41, srv1652031645=32, srv1051007714=1, srv1620181053=31, srv154686500=25, srv646747398=80, srv1399896240=18, srv265642781=63, srv2080464196=53, srv935925832=95, srv473084183=73, srv1269244740=13, srv1778883681=36, srv2061011169=51, srv1198735154=7, srv2045412648=49, srv884085010=91, srv282026324=64, srv2033217985=47, srv815387775=87, srv633533415=78, srv1227974848=9, srv199815715=45, srv123376764=12, srv985722840=98, srv354681993=69, srv703172514=82, srv1685301828=34, srv1965103883=42, srv942970412=96, srv905333270=93, srv838872599=89, srv2076464993=52, srv202481508=46, srv2052420381=50, srv69549737=81, srv1798728805=38, srv811608707=86, srv1662497634=33, srv402623599=72, srv1196718335=6, srv866077660=90, srv2096104151=54, srv55150560=77, srv160892268=28, srv1527681684=24, srv473736616=74, srv1028001532=0, srv300066544=66, srv1423488730=19, srv397535460=71, srv1614432340=30, srv770852387=84, srv969610978=97, srv930130903=94, srv1171503074=5, srv1727381088=35, srv2107816420=56, srv1990529924=44, srv752628968=83, srv1524005881=23, srv1611700567=29, srv257539047=62, srv998631095=99, srv1374238790=17} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-30 05:53:21,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-30 05:53:21,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-30 05:53:21,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-30 05:53:21,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-30 05:53:21,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-30 05:53:21,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-30 05:53:21,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-30 05:53:21,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-30 05:53:21,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-30 05:53:21,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-30 05:53:21,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-30 05:53:21,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-30 05:53:21,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-30 05:53:21,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-30 05:53:21,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-30 05:53:21,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-30 05:53:21,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-30 05:53:21,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-30 05:53:21,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-30 05:53:21,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-30 05:53:21,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-30 05:53:21,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-30 05:53:21,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-30 05:53:21,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-30 05:53:21,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-30 05:53:21,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-30 05:53:21,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-30 05:53:21,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-30 05:53:21,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-30 05:53:21,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-30 05:53:21,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-30 05:53:21,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-30 05:53:21,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-30 05:53:21,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-30 05:53:21,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-30 05:53:21,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-30 05:53:21,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-30 05:53:21,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-30 05:53:21,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-30 05:53:21,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-30 05:53:21,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-30 05:53:21,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-30 05:53:21,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-30 05:53:21,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-30 05:53:21,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-30 05:53:21,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-30 05:53:21,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-30 05:53:21,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-30 05:53:21,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-30 05:53:21,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-30 05:53:21,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-30 05:53:21,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-30 05:53:21,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-30 05:53:21,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-30 05:53:21,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-30 05:53:21,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-30 05:53:21,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-30 05:53:21,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-30 05:53:21,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-30 05:53:21,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-30 05:53:21,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-30 05:53:21,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-30 05:53:21,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-30 05:53:21,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-30 05:53:21,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-30 05:53:21,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-30 05:53:21,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-30 05:53:21,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-30 05:53:21,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-30 05:53:21,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-30 05:53:21,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-30 05:53:21,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-30 05:53:21,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-30 05:53:21,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-30 05:53:21,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-30 05:53:21,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-30 05:53:21,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-30 05:53:21,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-30 05:53:21,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-30 05:53:21,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-30 05:53:21,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-30 05:53:21,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-30 05:53:21,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-30 05:53:21,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-30 05:53:21,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-30 05:53:21,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-30 05:53:21,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-30 05:53:21,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-30 05:53:21,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-30 05:53:21,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-30 05:53:21,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-30 05:53:21,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-30 05:53:21,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-30 05:53:21,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-30 05:53:21,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-30 05:53:21,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-30 05:53:21,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-30 05:53:21,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-30 05:53:21,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-30 05:53:21,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-30 05:53:21,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-30 05:53:21,259 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02440921857343055); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.036203068617575, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02440921857343055); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04608333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-30 05:53:21,260 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-12-30 05:53:21,260 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.015157597578666807, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02440921857343055); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.036203068617575, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02440921857343055); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04608333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-12-30 05:53:21,611 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 455 ms to try 5000 different iterations.  Found a solution that moves 1002 regions; Going from a computed imbalance of 0.015157597578666807 to a new imbalance of 0.0010622781207016328. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008004550395605613); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00919913136971917); MoveCostFunction : (multiplier=7.0, imbalance=0.11133333333333334, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.008004550395605613); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.833333333333334E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.005); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-30 05:53:21,613 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1423488730:81 , srv2103801351:82 , srv237354130:82 , srv1374238790:83 , srv328035743:83 , srv397535460:83 , srv820246453:83 , srv1144242971:84 , srv1281502886:84 , srv1527681684:84 , srv1662497634:84 , srv178896665:84 , srv1977216623:84 , srv199815715:84 , srv250695334:84 , srv257539047:84 , srv265642781:84 , srv300066544:84 , srv304126051:84 , srv402623599:84 , srv473084183:84 , srv69549737:84 , srv815387775:84 , srv1072157603:85 , srv1171503074:85 , srv123376764:85 , srv1316015936:85 , srv1399896240:85 , srv147021237:85 , srv160892268:85 , srv1614432340:85 , srv1620181053:85 , srv1652031645:85 , srv1685301828:85 , srv1778883681:85 , srv1798728805:85 , srv1965103883:85 , srv202481508:85 , srv2061011169:85 , srv218976722:85 , srv473736616:85 , srv55150560:85 , srv703172514:85 , srv752628968:85 , srv811608707:85 , srv905333270:85 , srv930130903:85 , srv969610978:85 , srv985722840:85 , srv1196718335:86 , srv1198735154:86 , srv1213130142:86 , srv1227974848:86 , srv1231841844:86 , srv1232681931:86 , srv1269244740:86 , srv1290915992:86 , srv1498707409:86 , srv154686500:86 , srv1727381088:86 , srv1874290241:86 , srv189938520:86 , srv1990529924:86 , srv2033217985:86 , srv2045412648:86 , srv2076464993:86 , srv2080464196:86 , srv2096104151:86 , srv2110350468:86 , srv511787462:86 , srv633533415:86 , srv770852387:86 , srv787221743:86 , srv838872599:86 , srv866077660:86 , srv902727305:86 , srv935925832:86 , srv942970412:86 , srv998631095:86 , srv1028001532:87 , srv1051007714:87 , srv1524005881:87 , srv159872542:87 , srv1913532144:87 , srv2038769188:87 , srv2052420381:87 , srv2107816420:87 , srv299638338:87 , srv550298934:87 , srv645911735:87 , srv646747398:87 , srv1069642509:88 , srv1611700567:88 , srv354681993:88 , srv385150683:88 , srv884085010:88 , srv1478615615:89 , srv1597422589:89 , srv255990223:89 , srv282026324:538 }
2023-12-30 05:53:21,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-30 05:53:21,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1069642509=2, srv304126051=67, srv1072157603=3, srv1213130142=8, srv820246453=88, srv159872542=27, srv178896665=37, srv2038769188=48, srv511787462=75, srv1231841844=10, srv1290915992=15, srv147021237=20, srv1144242971=4, srv1597422589=26, srv1316015936=16, srv189938520=40, srv255990223=61, srv1281502886=14, srv1478615615=21, srv250695334=60, srv902727305=92, srv237354130=59, srv299638338=65, srv1232681931=11, srv328035743=68, srv787221743=85, srv385150683=70, srv218976722=58, srv1498707409=22, srv1874290241=39, srv2103801351=55, srv1977216623=43, srv2110350468=57, srv550298934=76, srv645911735=79, srv1913532144=41, srv1652031645=32, srv1051007714=1, srv1620181053=31, srv154686500=25, srv646747398=80, srv1399896240=18, srv265642781=63, srv2080464196=53, srv935925832=95, srv473084183=73, srv1269244740=13, srv1778883681=36, srv2061011169=51, srv1198735154=7, srv2045412648=49, srv884085010=91, srv282026324=64, srv2033217985=47, srv815387775=87, srv633533415=78, srv1227974848=9, srv199815715=45, srv123376764=12, srv985722840=98, srv354681993=69, srv703172514=82, srv1685301828=34, srv1965103883=42, srv942970412=96, srv905333270=93, srv838872599=89, srv2076464993=52, srv202481508=46, srv2052420381=50, srv69549737=81, srv1798728805=38, srv811608707=86, srv1662497634=33, srv402623599=72, srv1196718335=6, srv866077660=90, srv2096104151=54, srv55150560=77, srv160892268=28, srv1527681684=24, srv473736616=74, srv1028001532=0, srv300066544=66, srv1423488730=19, srv397535460=71, srv1614432340=30, srv770852387=84, srv969610978=97, srv930130903=94, srv1171503074=5, srv1727381088=35, srv2107816420=56, srv1990529924=44, srv752628968=83, srv1524005881=23, srv1611700567=29, srv257539047=62, srv998631095=99, srv1374238790=17} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-30 05:53:21,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-30 05:53:21,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-30 05:53:21,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-30 05:53:21,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-30 05:53:21,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-30 05:53:21,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-30 05:53:21,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-30 05:53:21,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-30 05:53:21,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-30 05:53:21,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-30 05:53:21,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-30 05:53:21,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-30 05:53:21,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-30 05:53:21,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-30 05:53:21,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-30 05:53:21,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-30 05:53:21,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-30 05:53:21,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-30 05:53:21,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-30 05:53:21,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-30 05:53:21,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-30 05:53:21,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-30 05:53:21,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-30 05:53:21,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-30 05:53:21,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-30 05:53:21,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-30 05:53:21,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-30 05:53:21,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-30 05:53:21,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-30 05:53:21,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-30 05:53:21,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-30 05:53:21,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-30 05:53:21,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-30 05:53:21,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-30 05:53:21,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-30 05:53:21,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-30 05:53:21,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-30 05:53:21,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-30 05:53:21,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-30 05:53:21,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-30 05:53:21,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-30 05:53:21,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-30 05:53:21,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-30 05:53:21,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-30 05:53:21,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-30 05:53:21,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-30 05:53:21,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-30 05:53:21,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-30 05:53:21,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-30 05:53:21,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-30 05:53:21,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-30 05:53:21,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-30 05:53:21,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-30 05:53:21,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-30 05:53:21,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-30 05:53:21,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-30 05:53:21,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-30 05:53:21,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-30 05:53:21,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-30 05:53:21,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-30 05:53:21,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-30 05:53:21,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-30 05:53:21,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-30 05:53:21,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-30 05:53:21,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-30 05:53:21,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-30 05:53:21,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-30 05:53:21,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-30 05:53:21,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-30 05:53:21,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-30 05:53:21,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-30 05:53:21,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-30 05:53:21,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-30 05:53:21,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-30 05:53:21,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-30 05:53:21,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-30 05:53:21,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-30 05:53:21,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-30 05:53:21,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-30 05:53:21,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-30 05:53:21,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-30 05:53:21,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-30 05:53:21,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-30 05:53:21,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-30 05:53:21,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-30 05:53:21,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-30 05:53:21,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-30 05:53:21,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-30 05:53:21,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-30 05:53:21,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-30 05:53:21,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-30 05:53:21,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-30 05:53:21,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-30 05:53:21,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-30 05:53:21,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-30 05:53:21,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-30 05:53:21,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-30 05:53:21,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-30 05:53:21,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-30 05:53:21,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-30 05:53:21,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-30 05:53:21,747 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008004550395605613); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00919913136971917); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.008004550395605613); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.833333333333334E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.005); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-30 05:53:21,747 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-12-30 05:53:21,748 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.0010552619667423672, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008004550395605613); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00919913136971917); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.008004550395605613); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.833333333333334E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.005); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-12-30 05:53:21,982 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 334 ms to try 5000 different iterations.  Found a solution that moves 318 regions; Going from a computed imbalance of 0.0010552619667423672 to a new imbalance of 4.417883497091089E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002118805753879094); MoveCostFunction : (multiplier=7.0, imbalance=0.035333333333333335, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-30 05:53:21,983 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1478615615:83 , srv1990529924:83 , srv646747398:83 , srv811608707:83 , srv942970412:83 , srv1069642509:84 , srv1171503074:84 , srv1196718335:84 , srv1213130142:84 , srv1231841844:84 , srv1290915992:84 , srv1374238790:84 , srv147021237:84 , srv1527681684:84 , srv1597422589:84 , srv1611700567:84 , srv1685301828:84 , srv1778883681:84 , srv178896665:84 , srv1798728805:84 , srv2038769188:84 , srv328035743:84 , srv385150683:84 , srv402623599:84 , srv550298934:84 , srv55150560:84 , srv770852387:84 , srv787221743:84 , srv866077660:84 , srv884085010:84 , srv905333270:84 , srv930130903:84 , srv935925832:84 , srv969610978:84 , srv998631095:84 , srv1028001532:85 , srv1072157603:85 , srv1198735154:85 , srv123376764:85 , srv1269244740:85 , srv1281502886:85 , srv1316015936:85 , srv1524005881:85 , srv154686500:85 , srv159872542:85 , srv160892268:85 , srv1614432340:85 , srv1652031645:85 , srv1662497634:85 , srv1727381088:85 , srv189938520:85 , srv1913532144:85 , srv1965103883:85 , srv1977216623:85 , srv199815715:85 , srv202481508:85 , srv2033217985:85 , srv2045412648:85 , srv2052420381:85 , srv2061011169:85 , srv2076464993:85 , srv2080464196:85 , srv2096104151:85 , srv2103801351:85 , srv2107816420:85 , srv2110350468:85 , srv218976722:85 , srv237354130:85 , srv250695334:85 , srv255990223:85 , srv257539047:85 , srv265642781:85 , srv299638338:85 , srv300066544:85 , srv354681993:85 , srv397535460:85 , srv473084183:85 , srv473736616:85 , srv511787462:85 , srv633533415:85 , srv645911735:85 , srv69549737:85 , srv703172514:85 , srv752628968:85 , srv815387775:85 , srv820246453:85 , srv838872599:85 , srv902727305:85 , srv1051007714:86 , srv1144242971:86 , srv1227974848:86 , srv1232681931:86 , srv1399896240:86 , srv1423488730:86 , srv1498707409:86 , srv1620181053:86 , srv1874290241:86 , srv985722840:86 , srv304126051:87 , srv282026324:613 }
2023-12-30 05:53:21,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-30 05:53:21,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1069642509=2, srv304126051=67, srv1072157603=3, srv1213130142=8, srv820246453=88, srv159872542=27, srv178896665=37, srv2038769188=48, srv511787462=75, srv1231841844=10, srv1290915992=15, srv147021237=20, srv1144242971=4, srv1597422589=26, srv1316015936=16, srv189938520=40, srv255990223=61, srv1281502886=14, srv1478615615=21, srv250695334=60, srv902727305=92, srv237354130=59, srv299638338=65, srv1232681931=11, srv328035743=68, srv787221743=85, srv385150683=70, srv218976722=58, srv1498707409=22, srv1874290241=39, srv2103801351=55, srv1977216623=43, srv2110350468=57, srv550298934=76, srv645911735=79, srv1913532144=41, srv1652031645=32, srv1051007714=1, srv1620181053=31, srv154686500=25, srv646747398=80, srv1399896240=18, srv265642781=63, srv2080464196=53, srv935925832=95, srv473084183=73, srv1269244740=13, srv1778883681=36, srv2061011169=51, srv1198735154=7, srv2045412648=49, srv884085010=91, srv282026324=64, srv2033217985=47, srv815387775=87, srv633533415=78, srv1227974848=9, srv199815715=45, srv123376764=12, srv985722840=98, srv354681993=69, srv703172514=82, srv1685301828=34, srv1965103883=42, srv942970412=96, srv905333270=93, srv838872599=89, srv2076464993=52, srv202481508=46, srv2052420381=50, srv69549737=81, srv1798728805=38, srv811608707=86, srv1662497634=33, srv402623599=72, srv1196718335=6, srv866077660=90, srv2096104151=54, srv55150560=77, srv160892268=28, srv1527681684=24, srv473736616=74, srv1028001532=0, srv300066544=66, srv1423488730=19, srv397535460=71, srv1614432340=30, srv770852387=84, srv969610978=97, srv930130903=94, srv1171503074=5, srv1727381088=35, srv2107816420=56, srv1990529924=44, srv752628968=83, srv1524005881=23, srv1611700567=29, srv257539047=62, srv998631095=99, srv1374238790=17} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-30 05:53:21,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-30 05:53:21,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-30 05:53:21,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-30 05:53:21,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-30 05:53:21,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-30 05:53:21,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-30 05:53:21,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-30 05:53:21,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-30 05:53:21,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-30 05:53:21,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-30 05:53:21,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-30 05:53:21,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-30 05:53:21,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-30 05:53:21,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-30 05:53:21,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-30 05:53:21,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-30 05:53:21,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-30 05:53:21,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-30 05:53:21,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-30 05:53:21,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-30 05:53:21,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-30 05:53:21,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-30 05:53:21,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-30 05:53:21,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-30 05:53:21,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-30 05:53:21,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-30 05:53:21,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-30 05:53:21,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-30 05:53:21,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-30 05:53:21,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-30 05:53:21,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-30 05:53:22,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-30 05:53:22,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-30 05:53:22,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-30 05:53:22,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-30 05:53:22,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-30 05:53:22,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-30 05:53:22,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-30 05:53:22,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-30 05:53:22,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-30 05:53:22,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-30 05:53:22,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-30 05:53:22,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-30 05:53:22,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-30 05:53:22,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-30 05:53:22,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-30 05:53:22,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-30 05:53:22,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-30 05:53:22,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-30 05:53:22,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-30 05:53:22,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-30 05:53:22,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-30 05:53:22,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-30 05:53:22,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-30 05:53:22,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-30 05:53:22,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-30 05:53:22,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-30 05:53:22,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-30 05:53:22,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-30 05:53:22,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-30 05:53:22,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-30 05:53:22,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-30 05:53:22,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-30 05:53:22,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-30 05:53:22,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-30 05:53:22,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-30 05:53:22,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-30 05:53:22,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-30 05:53:22,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-30 05:53:22,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-30 05:53:22,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-30 05:53:22,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-30 05:53:22,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-30 05:53:22,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-30 05:53:22,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-30 05:53:22,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-30 05:53:22,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-30 05:53:22,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-30 05:53:22,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-30 05:53:22,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-30 05:53:22,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-30 05:53:22,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-30 05:53:22,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-30 05:53:22,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-30 05:53:22,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-30 05:53:22,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-30 05:53:22,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-30 05:53:22,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-30 05:53:22,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-30 05:53:22,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-30 05:53:22,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-30 05:53:22,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-30 05:53:22,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-30 05:53:22,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-30 05:53:22,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-30 05:53:22,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-30 05:53:22,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-30 05:53:22,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-30 05:53:22,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-30 05:53:22,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-30 05:53:22,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-30 05:53:22,106 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002118805753879094); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-30 05:53:22,106 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-12-30 05:53:22,106 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=4.1952151379048186E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002118805753879094); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-12-30 05:53:22,340 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 329 ms to try 5000 different iterations.  Found a solution that moves 38 regions; Going from a computed imbalance of 4.1952151379048186E-5 to a new imbalance of 7.33932098340738E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0015713484026367722); MoveCostFunction : (multiplier=7.0, imbalance=0.004222222222222222); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-30 05:53:22,340 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1196718335:83 , srv1498707409:83 , srv1614432340:83 , srv1620181053:83 , srv1874290241:83 , srv255990223:83 , srv304126051:83 , srv397535460:83 , srv633533415:83 , srv985722840:83 , srv1028001532:84 , srv1051007714:84 , srv1069642509:84 , srv1072157603:84 , srv1144242971:84 , srv1171503074:84 , srv1198735154:84 , srv1213130142:84 , srv1227974848:84 , srv1231841844:84 , srv1232681931:84 , srv123376764:84 , srv1269244740:84 , srv1281502886:84 , srv1290915992:84 , srv1316015936:84 , srv1374238790:84 , srv1399896240:84 , srv1423488730:84 , srv147021237:84 , srv1478615615:84 , srv1524005881:84 , srv154686500:84 , srv1597422589:84 , srv159872542:84 , srv160892268:84 , srv1611700567:84 , srv1652031645:84 , srv1685301828:84 , srv1727381088:84 , srv1778883681:84 , srv178896665:84 , srv1798728805:84 , srv1913532144:84 , srv1965103883:84 , srv1977216623:84 , srv199815715:84 , srv202481508:84 , srv2033217985:84 , srv2038769188:84 , srv2045412648:84 , srv2052420381:84 , srv2061011169:84 , srv2076464993:84 , srv2080464196:84 , srv2096104151:84 , srv2103801351:84 , srv2107816420:84 , srv218976722:84 , srv237354130:84 , srv250695334:84 , srv257539047:84 , srv265642781:84 , srv299638338:84 , srv300066544:84 , srv354681993:84 , srv402623599:84 , srv473084183:84 , srv473736616:84 , srv511787462:84 , srv55150560:84 , srv645911735:84 , srv69549737:84 , srv703172514:84 , srv752628968:84 , srv770852387:84 , srv787221743:84 , srv811608707:84 , srv815387775:84 , srv820246453:84 , srv838872599:84 , srv866077660:84 , srv884085010:84 , srv902727305:84 , srv905333270:84 , srv930130903:84 , srv935925832:84 , srv969610978:84 , srv998631095:84 , srv1527681684:85 , srv1662497634:85 , srv189938520:85 , srv1990529924:85 , srv2110350468:85 , srv328035743:85 , srv385150683:85 , srv550298934:85 , srv646747398:85 , srv942970412:85 , srv282026324:684 }
2023-12-30 05:53:22,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-30 05:53:22,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1069642509=2, srv304126051=67, srv1072157603=3, srv1213130142=8, srv820246453=88, srv159872542=27, srv178896665=37, srv2038769188=48, srv511787462=75, srv1231841844=10, srv1290915992=15, srv147021237=20, srv1144242971=4, srv1597422589=26, srv1316015936=16, srv189938520=40, srv255990223=61, srv1281502886=14, srv1478615615=21, srv250695334=60, srv902727305=92, srv237354130=59, srv299638338=65, srv1232681931=11, srv328035743=68, srv787221743=85, srv385150683=70, srv218976722=58, srv1498707409=22, srv1874290241=39, srv2103801351=55, srv1977216623=43, srv2110350468=57, srv550298934=76, srv645911735=79, srv1913532144=41, srv1652031645=32, srv1051007714=1, srv1620181053=31, srv154686500=25, srv646747398=80, srv1399896240=18, srv265642781=63, srv2080464196=53, srv935925832=95, srv473084183=73, srv1269244740=13, srv1778883681=36, srv2061011169=51, srv1198735154=7, srv2045412648=49, srv884085010=91, srv282026324=64, srv2033217985=47, srv815387775=87, srv633533415=78, srv1227974848=9, srv199815715=45, srv123376764=12, srv985722840=98, srv354681993=69, srv703172514=82, srv1685301828=34, srv1965103883=42, srv942970412=96, srv905333270=93, srv838872599=89, srv2076464993=52, srv202481508=46, srv2052420381=50, srv69549737=81, srv1798728805=38, srv811608707=86, srv1662497634=33, srv402623599=72, srv1196718335=6, srv866077660=90, srv2096104151=54, srv55150560=77, srv160892268=28, srv1527681684=24, srv473736616=74, srv1028001532=0, srv300066544=66, srv1423488730=19, srv397535460=71, srv1614432340=30, srv770852387=84, srv969610978=97, srv930130903=94, srv1171503074=5, srv1727381088=35, srv2107816420=56, srv1990529924=44, srv752628968=83, srv1524005881=23, srv1611700567=29, srv257539047=62, srv998631095=99, srv1374238790=17} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-30 05:53:22,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-30 05:53:22,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-30 05:53:22,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-30 05:53:22,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-30 05:53:22,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-30 05:53:22,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-30 05:53:22,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-30 05:53:22,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-30 05:53:22,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-30 05:53:22,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-30 05:53:22,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-30 05:53:22,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-30 05:53:22,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-30 05:53:22,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-30 05:53:22,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-30 05:53:22,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-30 05:53:22,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-30 05:53:22,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-30 05:53:22,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-30 05:53:22,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-30 05:53:22,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-30 05:53:22,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-30 05:53:22,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-30 05:53:22,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-30 05:53:22,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-30 05:53:22,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-30 05:53:22,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-30 05:53:22,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-30 05:53:22,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-30 05:53:22,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-30 05:53:22,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-30 05:53:22,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-30 05:53:22,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-30 05:53:22,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-30 05:53:22,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-30 05:53:22,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-30 05:53:22,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-30 05:53:22,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-30 05:53:22,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-30 05:53:22,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-30 05:53:22,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-30 05:53:22,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-30 05:53:22,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-30 05:53:22,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-30 05:53:22,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-30 05:53:22,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-30 05:53:22,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-30 05:53:22,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-30 05:53:22,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-30 05:53:22,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-30 05:53:22,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-30 05:53:22,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-30 05:53:22,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-30 05:53:22,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-30 05:53:22,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-30 05:53:22,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-30 05:53:22,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-30 05:53:22,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-30 05:53:22,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-30 05:53:22,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-30 05:53:22,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-30 05:53:22,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-30 05:53:22,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-30 05:53:22,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-30 05:53:22,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-30 05:53:22,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-30 05:53:22,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-30 05:53:22,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-30 05:53:22,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-30 05:53:22,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-30 05:53:22,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-30 05:53:22,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-30 05:53:22,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-30 05:53:22,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-30 05:53:22,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-30 05:53:22,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-30 05:53:22,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-30 05:53:22,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-30 05:53:22,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-30 05:53:22,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-30 05:53:22,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-30 05:53:22,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-30 05:53:22,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-30 05:53:22,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-30 05:53:22,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-30 05:53:22,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-30 05:53:22,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-30 05:53:22,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-30 05:53:22,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-30 05:53:22,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-30 05:53:22,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-30 05:53:22,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-30 05:53:22,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-30 05:53:22,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-30 05:53:22,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-30 05:53:22,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-30 05:53:22,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-30 05:53:22,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-30 05:53:22,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-30 05:53:22,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-30 05:53:22,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-30 05:53:22,471 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=7.073239296329447E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0015713484026367722); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-30 05:53:22,471 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1196718335:83 , srv1498707409:83 , srv1614432340:83 , srv1620181053:83 , srv1874290241:83 , srv255990223:83 , srv304126051:83 , srv397535460:83 , srv633533415:83 , srv985722840:83 , srv1028001532:84 , srv1051007714:84 , srv1069642509:84 , srv1072157603:84 , srv1144242971:84 , srv1171503074:84 , srv1198735154:84 , srv1213130142:84 , srv1227974848:84 , srv1231841844:84 , srv1232681931:84 , srv123376764:84 , srv1269244740:84 , srv1281502886:84 , srv1290915992:84 , srv1316015936:84 , srv1374238790:84 , srv1399896240:84 , srv1423488730:84 , srv147021237:84 , srv1478615615:84 , srv1524005881:84 , srv154686500:84 , srv1597422589:84 , srv159872542:84 , srv160892268:84 , srv1611700567:84 , srv1652031645:84 , srv1685301828:84 , srv1727381088:84 , srv1778883681:84 , srv178896665:84 , srv1798728805:84 , srv1913532144:84 , srv1965103883:84 , srv1977216623:84 , srv199815715:84 , srv202481508:84 , srv2033217985:84 , srv2038769188:84 , srv2045412648:84 , srv2052420381:84 , srv2061011169:84 , srv2076464993:84 , srv2080464196:84 , srv2096104151:84 , srv2103801351:84 , srv2107816420:84 , srv218976722:84 , srv237354130:84 , srv250695334:84 , srv257539047:84 , srv265642781:84 , srv299638338:84 , srv300066544:84 , srv354681993:84 , srv402623599:84 , srv473084183:84 , srv473736616:84 , srv511787462:84 , srv55150560:84 , srv645911735:84 , srv69549737:84 , srv703172514:84 , srv752628968:84 , srv770852387:84 , srv787221743:84 , srv811608707:84 , srv815387775:84 , srv820246453:84 , srv838872599:84 , srv866077660:84 , srv884085010:84 , srv902727305:84 , srv905333270:84 , srv930130903:84 , srv935925832:84 , srv969610978:84 , srv998631095:84 , srv1527681684:85 , srv1662497634:85 , srv189938520:85 , srv1990529924:85 , srv2110350468:85 , srv328035743:85 , srv385150683:85 , srv550298934:85 , srv646747398:85 , srv942970412:85 , srv282026324:684 }
2023-12-30 05:53:22,506 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=262 (was 258) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 169), AvailableMemoryMB=4807 (was 5423)
2023-12-30 05:53:22,514 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=262, ProcessCount=169, AvailableMemoryMB=4807
2023-12-30 05:53:22,515 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-12-30 05:53:22,515 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-12-30 05:53:22,534 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1027483707:84 , srv1091777597:84 , srv1091936012:84 , srv1127388722:84 , srv1129769236:84 , srv130327095:84 , srv1308462032:84 , srv1364587187:84 , srv1421877914:84 , srv1503891909:84 , srv152589761:84 , srv1696575269:84 , srv1721896829:84 , srv1842025170:84 , srv1894520917:84 , srv1918529672:84 , srv1953227925:84 , srv2086606959:84 , srv2102742344:84 , srv278319397:84 , srv323510503:84 , srv397022118:84 , srv507968514:84 , srv521760809:84 , srv628695883:84 , srv641351061:84 , srv667995571:84 , srv807178387:84 , srv943892594:84 , srv419713325:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-12-30 05:53:22,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-30 05:53:22,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2102742344=18, srv1091936012=2, srv130327095=5, srv1129769236=4, srv1842025170=13, srv1421877914=8, srv2086606959=17, srv507968514=23, srv397022118=21, srv1721896829=12, srv1894520917=14, srv641351061=26, srv1696575269=11, srv667995571=27, srv943892594=29, srv419713325=22, srv1364587187=7, srv1027483707=0, srv1503891909=9, srv1918529672=15, srv521760809=24, srv807178387=28, srv628695883=25, srv1127388722=3, srv323510503=20, srv1308462032=6, srv1953227925=16, srv278319397=19, srv152589761=10, srv1091777597=1} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-30 05:53:22,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-30 05:53:22,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-30 05:53:22,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-30 05:53:22,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-30 05:53:22,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-30 05:53:22,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-30 05:53:22,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-30 05:53:22,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-30 05:53:22,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-30 05:53:22,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-30 05:53:22,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-30 05:53:22,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-30 05:53:22,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-30 05:53:22,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-30 05:53:22,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-30 05:53:22,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-30 05:53:22,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-30 05:53:22,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-30 05:53:22,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-30 05:53:22,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-30 05:53:22,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-30 05:53:22,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-30 05:53:22,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-30 05:53:22,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-30 05:53:22,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-30 05:53:22,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-30 05:53:22,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-30 05:53:22,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-30 05:53:22,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-30 05:53:22,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-30 05:53:22,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-12-30 05:53:22,571 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0982132801810032, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-30 05:53:22,571 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912502510103172, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0982132801810032, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-12-30 05:55:22,559 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5842938 different iterations.  Found a solution that moves 1856 regions; Going from a computed imbalance of 0.9912502510103172 to a new imbalance of 4.3319965896196796E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6874074074074074, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-30 05:55:22,565 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1027483707:90 , srv1091777597:90 , srv1091936012:90 , srv1127388722:90 , srv1129769236:90 , srv130327095:90 , srv1308462032:90 , srv1364587187:90 , srv1421877914:90 , srv1503891909:90 , srv152589761:90 , srv1696575269:90 , srv1721896829:90 , srv1842025170:90 , srv1894520917:90 , srv1918529672:90 , srv1953227925:90 , srv2086606959:90 , srv2102742344:90 , srv278319397:90 , srv323510503:90 , srv397022118:90 , srv419713325:90 , srv507968514:90 , srv521760809:90 , srv628695883:90 , srv641351061:90 , srv667995571:90 , srv807178387:90 , srv943892594:90 }
2023-12-30 05:55:22,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-30 05:55:22,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2102742344=18, srv1091936012=2, srv130327095=5, srv1129769236=4, srv1842025170=13, srv1421877914=8, srv2086606959=17, srv507968514=23, srv397022118=21, srv1721896829=12, srv1894520917=14, srv641351061=26, srv1696575269=11, srv667995571=27, srv943892594=29, srv419713325=22, srv1364587187=7, srv1027483707=0, srv1503891909=9, srv1918529672=15, srv521760809=24, srv807178387=28, srv628695883=25, srv1127388722=3, srv323510503=20, srv1308462032=6, srv1953227925=16, srv278319397=19, srv152589761=10, srv1091777597=1} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-30 05:55:22,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-30 05:55:22,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-30 05:55:22,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-30 05:55:22,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-30 05:55:22,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-30 05:55:22,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-30 05:55:22,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-30 05:55:22,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-30 05:55:22,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-30 05:55:22,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-30 05:55:22,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-30 05:55:22,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-30 05:55:22,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-30 05:55:22,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-30 05:55:22,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-30 05:55:22,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-30 05:55:22,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-30 05:55:22,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-30 05:55:22,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-30 05:55:22,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-30 05:55:22,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-30 05:55:22,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-30 05:55:22,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-30 05:55:22,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-30 05:55:22,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-30 05:55:22,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-30 05:55:22,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-30 05:55:22,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-30 05:55:22,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-30 05:55:22,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-30 05:55:22,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-12-30 05:55:22,592 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-30 05:55:22,592 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1027483707:90 , srv1091777597:90 , srv1091936012:90 , srv1127388722:90 , srv1129769236:90 , srv130327095:90 , srv1308462032:90 , srv1364587187:90 , srv1421877914:90 , srv1503891909:90 , srv152589761:90 , srv1696575269:90 , srv1721896829:90 , srv1842025170:90 , srv1894520917:90 , srv1918529672:90 , srv1953227925:90 , srv2086606959:90 , srv2102742344:90 , srv278319397:90 , srv323510503:90 , srv397022118:90 , srv419713325:90 , srv507968514:90 , srv521760809:90 , srv628695883:90 , srv641351061:90 , srv667995571:90 , srv807178387:90 , srv943892594:90 }
2023-12-30 05:55:22,609 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=173 (was 262), ProcessCount=167 (was 169), AvailableMemoryMB=4465 (was 4807)