2023-11-02 15:55:48,071 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-11-02 15:55:48,322 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-11-02 15:55:48,438 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-11-02 15:55:48,527 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-02 15:55:48,535 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-11-02 15:55:48,552 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=501, ProcessCount=179, AvailableMemoryMB=2200
2023-11-02 15:55:48,554 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-02 15:55:48,554 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-11-02 15:55:48,674 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1023726577:84 , srv1072250486:84 , srv1074668308:84 , srv110345042:84 , srv1137693895:84 , srv1191041757:84 , srv1197373626:84 , srv1234083891:84 , srv1252325856:84 , srv1272206980:84 , srv1284672376:84 , srv1329432781:84 , srv1360916389:84 , srv1385974267:84 , srv139098792:84 , srv1406576910:84 , srv1408532845:84 , srv1410186028:84 , srv1447794946:84 , srv1476916941:84 , srv1498132383:84 , srv1501007099:84 , srv1503368570:84 , srv1582060181:84 , srv1585830226:84 , srv1639757848:84 , srv1663443158:84 , srv1709007636:84 , srv1730221160:84 , srv177789160:84 , srv1787020744:84 , srv1787454732:84 , srv1807640493:84 , srv1828587780:84 , srv1862489435:84 , srv1865003625:84 , srv1879724488:84 , srv188069612:84 , srv1911872306:84 , srv1912636597:84 , srv1948149954:84 , srv1953141633:84 , srv1961997052:84 , srv1976631629:84 , srv1982585649:84 , srv1993640754:84 , srv1999429103:84 , srv2004227232:84 , srv2045598307:84 , srv2048707461:84 , srv205126670:84 , srv2137807933:84 , srv218190829:84 , srv237101174:84 , srv248852011:84 , srv255177449:84 , srv284045326:84 , srv372702083:84 , srv377319443:84 , srv401169459:84 , srv424521173:84 , srv480549382:84 , srv48626042:84 , srv518731295:84 , srv519399986:84 , srv520593614:84 , srv526053339:84 , srv539220227:84 , srv543382739:84 , srv549149739:84 , srv565767614:84 , srv584568139:84 , srv59988934:84 , srv603413215:84 , srv609382280:84 , srv619899731:84 , srv643571435:84 , srv646062640:84 , srv652982955:84 , srv661365551:84 , srv737280100:84 , srv738634588:84 , srv754162017:84 , srv772759390:84 , srv776574302:84 , srv779283332:84 , srv793417979:84 , srv80345466:84 , srv815443750:84 , srv830434270:84 , srv831594694:84 , srv865719484:84 , srv872082844:84 , srv875693683:84 , srv876922201:84 , srv899463815:84 , srv906258380:84 , srv959364053:84 , srv991964466:84 , srv1405991983:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-11-02 15:55:48,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-02 15:55:48,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1137693895=4, srv1408532845=17, srv1410186028=18, srv1197373626=6, srv80345466=88, srv1498132383=21, srv59988934=73, srv1385974267=13, srv1406576910=16, srv609382280=75, srv1948149954=41, srv1272206980=9, srv876922201=95, srv1191041757=5, srv661365551=80, srv772759390=84, srv1585830226=25, srv603413215=74, srv188069612=38, srv815443750=89, srv1807640493=33, srv652982955=79, srv738634588=82, srv139098792=14, srv520593614=66, srv518731295=64, srv1730221160=29, srv906258380=97, srv1072250486=1, srv793417979=87, srv754162017=83, srv991964466=99, srv218190829=53, srv177789160=30, srv1582060181=24, srv1405991983=15, srv1993640754=46, srv1503368570=23, srv1879724488=37, srv401169459=60, srv1982585649=45, srv584568139=72, srv1284672376=10, srv1709007636=28, srv248852011=55, srv959364053=98, srv1787020744=31, srv1828587780=34, srv519399986=65, srv1911872306=39, srv875693683=94, srv2004227232=48, srv737280100=81, srv526053339=67, srv1961997052=43, srv1862489435=35, srv237101174=54, srv424521173=61, srv776574302=85, srv865719484=92, srv1639757848=26, srv1787454732=32, srv899463815=96, srv205126670=51, srv1329432781=11, srv1999429103=47, srv549149739=70, srv830434270=90, srv1663443158=27, srv2045598307=49, srv1234083891=7, srv284045326=57, srv1023726577=0, srv372702083=58, srv1476916941=20, srv1976631629=44, srv779283332=86, srv831594694=91, srv646062640=78, srv1074668308=2, srv543382739=69, srv1912636597=40, srv1501007099=22, srv1252325856=8, srv619899731=76, srv643571435=77, srv1953141633=42, srv255177449=56, srv1360916389=12, srv1447794946=19, srv2137807933=52, srv110345042=3, srv2048707461=50, srv872082844=93, srv480549382=62, srv1865003625=36, srv377319443=59, srv539220227=68, srv48626042=63, srv565767614=71} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-02 15:55:48,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-02 15:55:48,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-02 15:55:48,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-02 15:55:48,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-02 15:55:48,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-02 15:55:48,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-02 15:55:48,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-02 15:55:48,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-02 15:55:48,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-02 15:55:48,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-02 15:55:48,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-02 15:55:48,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-02 15:55:48,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-02 15:55:48,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-02 15:55:48,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-02 15:55:48,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-02 15:55:48,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-02 15:55:48,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-02 15:55:48,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-02 15:55:48,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-02 15:55:48,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-02 15:55:48,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-02 15:55:48,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-02 15:55:48,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-02 15:55:48,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-02 15:55:48,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-02 15:55:48,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-02 15:55:48,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-02 15:55:48,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-02 15:55:48,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-02 15:55:48,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-02 15:55:48,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-02 15:55:48,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-02 15:55:48,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-02 15:55:48,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-02 15:55:48,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-02 15:55:48,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-02 15:55:48,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-02 15:55:48,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-02 15:55:48,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-02 15:55:48,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-02 15:55:48,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-02 15:55:48,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-02 15:55:48,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-02 15:55:48,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-02 15:55:48,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-02 15:55:48,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-02 15:55:48,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-02 15:55:48,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-02 15:55:48,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-02 15:55:48,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-02 15:55:48,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-02 15:55:48,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-02 15:55:48,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-02 15:55:48,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-02 15:55:48,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-02 15:55:48,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-02 15:55:48,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-02 15:55:48,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-02 15:55:48,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-02 15:55:48,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-02 15:55:48,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-02 15:55:48,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-02 15:55:48,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-02 15:55:48,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-02 15:55:48,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-02 15:55:48,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-02 15:55:48,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-02 15:55:48,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-02 15:55:48,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-02 15:55:48,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-02 15:55:48,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-02 15:55:48,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-02 15:55:48,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-02 15:55:48,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-02 15:55:48,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-02 15:55:48,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-02 15:55:48,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-02 15:55:48,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-02 15:55:48,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-02 15:55:48,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-02 15:55:48,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-02 15:55:48,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-02 15:55:48,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-02 15:55:48,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-02 15:55:48,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-02 15:55:48,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-02 15:55:48,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-02 15:55:48,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-02 15:55:48,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-02 15:55:48,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-02 15:55:48,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-02 15:55:48,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-02 15:55:48,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-02 15:55:48,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-02 15:55:48,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-02 15:55:48,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-02 15:55:48,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-02 15:55:48,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-02 15:55:48,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-02 15:55:48,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-02 15:55:49,078 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-02 15:55:49,078 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-02 15:55:49,078 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-02 15:55:49,589 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 795 ms to try 5000 different iterations.  Found a solution that moves 4720 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17385725738305488. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.041058752717417796, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.058476735711978825, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5244444444444445, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.041058752717417796, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.16875, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23816666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-02 15:55:49,599 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv2004227232:82 , srv2048707461:82 , srv1072250486:83 , srv1360916389:83 , srv1585830226:83 , srv1953141633:83 , srv1976631629:83 , srv480549382:83 , srv619899731:83 , srv661365551:83 , srv80345466:83 , srv815443750:83 , srv865719484:83 , srv876922201:83 , srv1408532845:84 , srv1730221160:84 , srv2045598307:84 , srv218190829:84 , srv284045326:84 , srv372702083:84 , srv603413215:84 , srv754162017:84 , srv776574302:84 , srv1137693895:85 , srv1191041757:85 , srv1252325856:85 , srv1284672376:85 , srv1410186028:85 , srv1498132383:85 , srv1503368570:85 , srv1709007636:85 , srv1787454732:85 , srv1993640754:85 , srv255177449:85 , srv539220227:85 , srv643571435:85 , srv652982955:85 , srv772759390:85 , srv779283332:85 , srv793417979:85 , srv1074668308:86 , srv1807640493:86 , srv1828587780:86 , srv1865003625:86 , srv1911872306:86 , srv401169459:86 , srv518731295:86 , srv519399986:86 , srv549149739:86 , srv59988934:86 , srv875693683:86 , srv959364053:86 , srv1023726577:87 , srv110345042:87 , srv1272206980:87 , srv1406576910:87 , srv1447794946:87 , srv1501007099:87 , srv1582060181:87 , srv1639757848:87 , srv1663443158:87 , srv1787020744:87 , srv1862489435:87 , srv1912636597:87 , srv1948149954:87 , srv1999429103:87 , srv2137807933:87 , srv424521173:87 , srv526053339:87 , srv646062640:87 , srv830434270:87 , srv906258380:87 , srv1234083891:88 , srv139098792:88 , srv1476916941:88 , srv1879724488:88 , srv1961997052:88 , srv1982585649:88 , srv205126670:88 , srv237101174:88 , srv48626042:88 , srv520593614:88 , srv543382739:88 , srv831594694:88 , srv872082844:88 , srv899463815:88 , srv1329432781:89 , srv377319443:89 , srv584568139:89 , srv609382280:89 , srv737280100:89 , srv738634588:89 , srv1385974267:90 , srv177789160:91 , srv188069612:91 , srv1197373626:92 , srv991964466:92 , srv565767614:94 , srv248852011:96 , srv1405991983:455 }
2023-11-02 15:55:49,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-02 15:55:49,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1137693895=4, srv1408532845=17, srv1410186028=18, srv1197373626=6, srv80345466=88, srv1498132383=21, srv59988934=73, srv1385974267=13, srv1406576910=16, srv609382280=75, srv1948149954=41, srv1272206980=9, srv876922201=95, srv1191041757=5, srv661365551=80, srv772759390=84, srv1585830226=25, srv603413215=74, srv188069612=38, srv815443750=89, srv1807640493=33, srv652982955=79, srv738634588=82, srv139098792=14, srv520593614=66, srv518731295=64, srv1730221160=29, srv906258380=97, srv1072250486=1, srv793417979=87, srv754162017=83, srv991964466=99, srv218190829=53, srv177789160=30, srv1582060181=24, srv1405991983=15, srv1993640754=46, srv1503368570=23, srv1879724488=37, srv401169459=60, srv1982585649=45, srv584568139=72, srv1284672376=10, srv1709007636=28, srv248852011=55, srv959364053=98, srv1787020744=31, srv1828587780=34, srv519399986=65, srv1911872306=39, srv875693683=94, srv2004227232=48, srv737280100=81, srv526053339=67, srv1961997052=43, srv1862489435=35, srv237101174=54, srv424521173=61, srv776574302=85, srv865719484=92, srv1639757848=26, srv1787454732=32, srv899463815=96, srv205126670=51, srv1329432781=11, srv1999429103=47, srv549149739=70, srv830434270=90, srv1663443158=27, srv2045598307=49, srv1234083891=7, srv284045326=57, srv1023726577=0, srv372702083=58, srv1476916941=20, srv1976631629=44, srv779283332=86, srv831594694=91, srv646062640=78, srv1074668308=2, srv543382739=69, srv1912636597=40, srv1501007099=22, srv1252325856=8, srv619899731=76, srv643571435=77, srv1953141633=42, srv255177449=56, srv1360916389=12, srv1447794946=19, srv2137807933=52, srv110345042=3, srv2048707461=50, srv872082844=93, srv480549382=62, srv1865003625=36, srv377319443=59, srv539220227=68, srv48626042=63, srv565767614=71} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-02 15:55:49,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-02 15:55:49,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-02 15:55:49,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-02 15:55:49,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-02 15:55:49,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-02 15:55:49,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-02 15:55:49,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-02 15:55:49,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-02 15:55:49,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-02 15:55:49,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-02 15:55:49,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-02 15:55:49,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-02 15:55:49,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-02 15:55:49,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-02 15:55:49,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-02 15:55:49,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-02 15:55:49,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-02 15:55:49,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-02 15:55:49,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-02 15:55:49,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-02 15:55:49,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-02 15:55:49,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-02 15:55:49,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-02 15:55:49,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-02 15:55:49,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-02 15:55:49,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-02 15:55:49,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-02 15:55:49,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-02 15:55:49,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-02 15:55:49,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-02 15:55:49,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-02 15:55:49,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-02 15:55:49,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-02 15:55:49,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-02 15:55:49,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-02 15:55:49,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-02 15:55:49,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-02 15:55:49,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-02 15:55:49,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-02 15:55:49,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-02 15:55:49,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-02 15:55:49,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-02 15:55:49,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-02 15:55:49,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-02 15:55:49,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-02 15:55:49,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-02 15:55:49,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-02 15:55:49,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-02 15:55:49,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-02 15:55:49,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-02 15:55:49,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-02 15:55:49,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-02 15:55:49,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-02 15:55:49,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-02 15:55:49,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-02 15:55:49,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-02 15:55:49,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-02 15:55:49,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-02 15:55:49,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-02 15:55:49,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-02 15:55:49,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-02 15:55:49,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-02 15:55:49,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-02 15:55:49,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-02 15:55:49,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-02 15:55:49,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-02 15:55:49,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-02 15:55:49,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-02 15:55:49,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-02 15:55:49,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-02 15:55:49,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-02 15:55:49,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-02 15:55:49,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-02 15:55:49,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-02 15:55:49,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-02 15:55:49,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-02 15:55:49,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-02 15:55:49,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-02 15:55:49,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-02 15:55:49,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-02 15:55:49,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-02 15:55:49,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-02 15:55:49,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-02 15:55:49,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-02 15:55:49,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-02 15:55:49,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-02 15:55:49,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-02 15:55:49,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-02 15:55:49,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-02 15:55:49,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-02 15:55:49,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-02 15:55:49,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-02 15:55:49,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-02 15:55:49,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-02 15:55:49,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-02 15:55:49,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-02 15:55:49,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-02 15:55:49,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-02 15:55:49,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-02 15:55:49,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-02 15:55:49,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-02 15:55:49,775 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.041058752717417796, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.058476735711978825, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.041058752717417796, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.16875, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23816666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-02 15:55:49,775 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-02 15:55:49,775 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17382420723665995, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.041058752717417796, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.058476735711978825, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.041058752717417796, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.16875, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23816666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-02 15:55:49,978 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 321 ms to try 5000 different iterations.  Found a solution that moves 2598 regions; Going from a computed imbalance of 0.17382420723665995 to a new imbalance of 0.013224281366184986. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02294177364932428); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039237971485349146, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2886666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02294177364932428); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010083333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.042666666666666665, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-02 15:55:49,988 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv565767614:77 , srv248852011:78 , srv1197373626:79 , srv177789160:81 , srv188069612:81 , srv991964466:81 , srv1385974267:82 , srv139098792:82 , srv1476916941:82 , srv1639757848:83 , srv2137807933:83 , srv48626042:83 , srv543382739:83 , srv584568139:83 , srv738634588:83 , srv1023726577:84 , srv1234083891:84 , srv1272206980:84 , srv1447794946:84 , srv1503368570:84 , srv1862489435:84 , srv1879724488:84 , srv237101174:84 , srv377319443:84 , srv526053339:84 , srv899463815:84 , srv110345042:85 , srv1329432781:85 , srv1406576910:85 , srv1410186028:85 , srv1501007099:85 , srv1582060181:85 , srv1787020744:85 , srv1807640493:85 , srv1828587780:85 , srv1948149954:85 , srv1961997052:85 , srv1982585649:85 , srv1999429103:85 , srv255177449:85 , srv424521173:85 , srv520593614:85 , srv539220227:85 , srv609382280:85 , srv646062640:85 , srv772759390:85 , srv906258380:85 , srv959364053:85 , srv1074668308:86 , srv1137693895:86 , srv1498132383:86 , srv1709007636:86 , srv1911872306:86 , srv1912636597:86 , srv1993640754:86 , srv205126670:86 , srv401169459:86 , srv480549382:86 , srv519399986:86 , srv59988934:86 , srv652982955:86 , srv793417979:86 , srv830434270:86 , srv831594694:86 , srv1191041757:87 , srv1252325856:87 , srv1360916389:87 , srv1408532845:87 , srv1663443158:87 , srv1730221160:87 , srv1865003625:87 , srv372702083:87 , srv518731295:87 , srv549149739:87 , srv643571435:87 , srv776574302:87 , srv779283332:87 , srv815443750:87 , srv872082844:87 , srv1284672376:88 , srv1787454732:88 , srv218190829:88 , srv284045326:88 , srv737280100:88 , srv754162017:88 , srv865719484:88 , srv1953141633:89 , srv1976631629:89 , srv2045598307:89 , srv603413215:89 , srv661365551:89 , srv875693683:89 , srv876922201:89 , srv1072250486:90 , srv1585830226:90 , srv2004227232:90 , srv80345466:90 , srv2048707461:91 , srv619899731:91 , srv1405991983:523 }
2023-11-02 15:55:49,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-02 15:55:49,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1137693895=4, srv1408532845=17, srv1410186028=18, srv1197373626=6, srv80345466=88, srv1498132383=21, srv59988934=73, srv1385974267=13, srv1406576910=16, srv609382280=75, srv1948149954=41, srv1272206980=9, srv876922201=95, srv1191041757=5, srv661365551=80, srv772759390=84, srv1585830226=25, srv603413215=74, srv188069612=38, srv815443750=89, srv1807640493=33, srv652982955=79, srv738634588=82, srv139098792=14, srv520593614=66, srv518731295=64, srv1730221160=29, srv906258380=97, srv1072250486=1, srv793417979=87, srv754162017=83, srv991964466=99, srv218190829=53, srv177789160=30, srv1582060181=24, srv1405991983=15, srv1993640754=46, srv1503368570=23, srv1879724488=37, srv401169459=60, srv1982585649=45, srv584568139=72, srv1284672376=10, srv1709007636=28, srv248852011=55, srv959364053=98, srv1787020744=31, srv1828587780=34, srv519399986=65, srv1911872306=39, srv875693683=94, srv2004227232=48, srv737280100=81, srv526053339=67, srv1961997052=43, srv1862489435=35, srv237101174=54, srv424521173=61, srv776574302=85, srv865719484=92, srv1639757848=26, srv1787454732=32, srv899463815=96, srv205126670=51, srv1329432781=11, srv1999429103=47, srv549149739=70, srv830434270=90, srv1663443158=27, srv2045598307=49, srv1234083891=7, srv284045326=57, srv1023726577=0, srv372702083=58, srv1476916941=20, srv1976631629=44, srv779283332=86, srv831594694=91, srv646062640=78, srv1074668308=2, srv543382739=69, srv1912636597=40, srv1501007099=22, srv1252325856=8, srv619899731=76, srv643571435=77, srv1953141633=42, srv255177449=56, srv1360916389=12, srv1447794946=19, srv2137807933=52, srv110345042=3, srv2048707461=50, srv872082844=93, srv480549382=62, srv1865003625=36, srv377319443=59, srv539220227=68, srv48626042=63, srv565767614=71} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-02 15:55:50,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-02 15:55:50,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-02 15:55:50,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-02 15:55:50,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-02 15:55:50,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-02 15:55:50,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-02 15:55:50,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-02 15:55:50,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-02 15:55:50,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-02 15:55:50,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-02 15:55:50,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-02 15:55:50,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-02 15:55:50,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-02 15:55:50,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-02 15:55:50,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-02 15:55:50,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-02 15:55:50,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-02 15:55:50,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-02 15:55:50,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-02 15:55:50,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-02 15:55:50,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-02 15:55:50,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-02 15:55:50,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-02 15:55:50,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-02 15:55:50,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-02 15:55:50,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-02 15:55:50,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-02 15:55:50,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-02 15:55:50,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-02 15:55:50,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-02 15:55:50,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-02 15:55:50,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-02 15:55:50,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-02 15:55:50,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-02 15:55:50,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-02 15:55:50,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-02 15:55:50,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-02 15:55:50,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-02 15:55:50,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-02 15:55:50,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-02 15:55:50,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-02 15:55:50,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-02 15:55:50,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-02 15:55:50,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-02 15:55:50,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-02 15:55:50,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-02 15:55:50,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-02 15:55:50,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-02 15:55:50,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-02 15:55:50,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-02 15:55:50,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-02 15:55:50,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-02 15:55:50,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-02 15:55:50,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-02 15:55:50,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-02 15:55:50,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-02 15:55:50,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-02 15:55:50,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-02 15:55:50,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-02 15:55:50,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-02 15:55:50,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-02 15:55:50,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-02 15:55:50,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-02 15:55:50,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-02 15:55:50,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-02 15:55:50,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-02 15:55:50,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-02 15:55:50,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-02 15:55:50,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-02 15:55:50,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-02 15:55:50,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-02 15:55:50,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-02 15:55:50,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-02 15:55:50,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-02 15:55:50,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-02 15:55:50,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-02 15:55:50,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-02 15:55:50,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-02 15:55:50,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-02 15:55:50,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-02 15:55:50,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-02 15:55:50,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-02 15:55:50,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-02 15:55:50,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-02 15:55:50,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-02 15:55:50,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-02 15:55:50,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-02 15:55:50,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-02 15:55:50,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-02 15:55:50,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-02 15:55:50,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-02 15:55:50,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-02 15:55:50,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-02 15:55:50,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-02 15:55:50,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-02 15:55:50,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-02 15:55:50,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-02 15:55:50,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-02 15:55:50,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-02 15:55:50,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-02 15:55:50,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-02 15:55:50,141 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02294177364932428); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039237971485349146, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02294177364932428); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010083333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.042666666666666665, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-02 15:55:50,142 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-02 15:55:50,142 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.013206089781368446, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02294177364932428); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039237971485349146, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02294177364932428); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010083333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.042666666666666665, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-02 15:55:50,326 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 281 ms to try 5000 different iterations.  Found a solution that moves 929 regions; Going from a computed imbalance of 0.013206089781368446 to a new imbalance of 7.980220720325154E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008138960231508756); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.013796663498152624); MoveCostFunction : (multiplier=7.0, imbalance=0.10322222222222223, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.008138960231508756); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.005166666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-02 15:55:50,328 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv737280100:81 , srv619899731:82 , srv1072250486:83 , srv1329432781:83 , srv1982585649:83 , srv205126670:83 , srv872082844:83 , srv991964466:83 , srv1284672376:84 , srv1585830226:84 , srv1879724488:84 , srv1961997052:84 , srv1976631629:84 , srv2045598307:84 , srv2048707461:84 , srv248852011:84 , srv401169459:84 , srv518731295:84 , srv520593614:84 , srv603413215:84 , srv830434270:84 , srv875693683:84 , srv1191041757:85 , srv1234083891:85 , srv1252325856:85 , srv1385974267:85 , srv1406576910:85 , srv1501007099:85 , srv1582060181:85 , srv1663443158:85 , srv1709007636:85 , srv1730221160:85 , srv177789160:85 , srv1787454732:85 , srv1828587780:85 , srv1865003625:85 , srv1911872306:85 , srv1912636597:85 , srv1953141633:85 , srv1993640754:85 , srv1999429103:85 , srv2004227232:85 , srv284045326:85 , srv377319443:85 , srv424521173:85 , srv519399986:85 , srv543382739:85 , srv549149739:85 , srv584568139:85 , srv609382280:85 , srv643571435:85 , srv661365551:85 , srv776574302:85 , srv80345466:85 , srv831594694:85 , srv865719484:85 , srv899463815:85 , srv906258380:85 , srv1023726577:86 , srv1074668308:86 , srv110345042:86 , srv1137693895:86 , srv1197373626:86 , srv1272206980:86 , srv1447794946:86 , srv1498132383:86 , srv1787020744:86 , srv1807640493:86 , srv1862489435:86 , srv188069612:86 , srv1948149954:86 , srv218190829:86 , srv255177449:86 , srv48626042:86 , srv526053339:86 , srv565767614:86 , srv59988934:86 , srv652982955:86 , srv738634588:86 , srv754162017:86 , srv815443750:86 , srv876922201:86 , srv1360916389:87 , srv139098792:87 , srv1408532845:87 , srv1476916941:87 , srv2137807933:87 , srv237101174:87 , srv372702083:87 , srv539220227:87 , srv779283332:87 , srv793417979:87 , srv959364053:87 , srv1410186028:88 , srv1639757848:88 , srv480549382:88 , srv772759390:88 , srv1503368570:89 , srv646062640:89 , srv1405991983:552 }
2023-11-02 15:55:50,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-02 15:55:50,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1137693895=4, srv1408532845=17, srv1410186028=18, srv1197373626=6, srv80345466=88, srv1498132383=21, srv59988934=73, srv1385974267=13, srv1406576910=16, srv609382280=75, srv1948149954=41, srv1272206980=9, srv876922201=95, srv1191041757=5, srv661365551=80, srv772759390=84, srv1585830226=25, srv603413215=74, srv188069612=38, srv815443750=89, srv1807640493=33, srv652982955=79, srv738634588=82, srv139098792=14, srv520593614=66, srv518731295=64, srv1730221160=29, srv906258380=97, srv1072250486=1, srv793417979=87, srv754162017=83, srv991964466=99, srv218190829=53, srv177789160=30, srv1582060181=24, srv1405991983=15, srv1993640754=46, srv1503368570=23, srv1879724488=37, srv401169459=60, srv1982585649=45, srv584568139=72, srv1284672376=10, srv1709007636=28, srv248852011=55, srv959364053=98, srv1787020744=31, srv1828587780=34, srv519399986=65, srv1911872306=39, srv875693683=94, srv2004227232=48, srv737280100=81, srv526053339=67, srv1961997052=43, srv1862489435=35, srv237101174=54, srv424521173=61, srv776574302=85, srv865719484=92, srv1639757848=26, srv1787454732=32, srv899463815=96, srv205126670=51, srv1329432781=11, srv1999429103=47, srv549149739=70, srv830434270=90, srv1663443158=27, srv2045598307=49, srv1234083891=7, srv284045326=57, srv1023726577=0, srv372702083=58, srv1476916941=20, srv1976631629=44, srv779283332=86, srv831594694=91, srv646062640=78, srv1074668308=2, srv543382739=69, srv1912636597=40, srv1501007099=22, srv1252325856=8, srv619899731=76, srv643571435=77, srv1953141633=42, srv255177449=56, srv1360916389=12, srv1447794946=19, srv2137807933=52, srv110345042=3, srv2048707461=50, srv872082844=93, srv480549382=62, srv1865003625=36, srv377319443=59, srv539220227=68, srv48626042=63, srv565767614=71} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-02 15:55:50,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-02 15:55:50,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-02 15:55:50,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-02 15:55:50,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-02 15:55:50,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-02 15:55:50,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-02 15:55:50,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-02 15:55:50,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-02 15:55:50,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-02 15:55:50,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-02 15:55:50,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-02 15:55:50,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-02 15:55:50,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-02 15:55:50,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-02 15:55:50,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-02 15:55:50,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-02 15:55:50,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-02 15:55:50,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-02 15:55:50,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-02 15:55:50,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-02 15:55:50,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-02 15:55:50,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-02 15:55:50,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-02 15:55:50,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-02 15:55:50,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-02 15:55:50,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-02 15:55:50,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-02 15:55:50,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-02 15:55:50,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-02 15:55:50,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-02 15:55:50,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-02 15:55:50,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-02 15:55:50,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-02 15:55:50,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-02 15:55:50,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-02 15:55:50,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-02 15:55:50,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-02 15:55:50,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-02 15:55:50,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-02 15:55:50,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-02 15:55:50,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-02 15:55:50,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-02 15:55:50,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-02 15:55:50,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-02 15:55:50,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-02 15:55:50,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-02 15:55:50,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-02 15:55:50,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-02 15:55:50,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-02 15:55:50,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-02 15:55:50,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-02 15:55:50,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-02 15:55:50,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-02 15:55:50,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-02 15:55:50,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-02 15:55:50,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-02 15:55:50,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-02 15:55:50,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-02 15:55:50,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-02 15:55:50,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-02 15:55:50,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-02 15:55:50,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-02 15:55:50,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-02 15:55:50,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-02 15:55:50,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-02 15:55:50,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-02 15:55:50,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-02 15:55:50,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-02 15:55:50,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-02 15:55:50,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-02 15:55:50,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-02 15:55:50,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-02 15:55:50,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-02 15:55:50,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-02 15:55:50,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-02 15:55:50,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-02 15:55:50,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-02 15:55:50,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-02 15:55:50,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-02 15:55:50,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-02 15:55:50,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-02 15:55:50,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-02 15:55:50,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-02 15:55:50,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-02 15:55:50,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-02 15:55:50,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-02 15:55:50,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-02 15:55:50,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-02 15:55:50,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-02 15:55:50,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-02 15:55:50,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-02 15:55:50,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-02 15:55:50,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-02 15:55:50,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-02 15:55:50,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-02 15:55:50,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-02 15:55:50,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-02 15:55:50,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-02 15:55:50,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-02 15:55:50,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-02 15:55:50,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-02 15:55:50,445 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008138960231508756); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.013796663498152624); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.008138960231508756); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.005166666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-02 15:55:50,445 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-02 15:55:50,445 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=7.915170749984259E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008138960231508756); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.013796663498152624); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.008138960231508756); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.005166666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-02 15:55:50,698 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 333 ms to try 5000 different iterations.  Found a solution that moves 359 regions; Going from a computed imbalance of 7.915170749984259E-4 to a new imbalance of 1.4223139542369792E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0021711298242630987); MoveCostFunction : (multiplier=7.0, imbalance=0.03988888888888889, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=5.833333333333334E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-02 15:55:50,699 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv609382280:82 , srv646062640:82 , srv1663443158:83 , srv237101174:83 , srv779283332:83 , srv1074668308:84 , srv110345042:84 , srv1197373626:84 , srv1408532845:84 , srv1410186028:84 , srv1498132383:84 , srv1503368570:84 , srv1585830226:84 , srv1639757848:84 , srv1787020744:84 , srv1787454732:84 , srv1865003625:84 , srv188069612:84 , srv1912636597:84 , srv1948149954:84 , srv1999429103:84 , srv218190829:84 , srv248852011:84 , srv372702083:84 , srv377319443:84 , srv48626042:84 , srv526053339:84 , srv549149739:84 , srv59988934:84 , srv652982955:84 , srv737280100:84 , srv772759390:84 , srv793417979:84 , srv80345466:84 , srv831594694:84 , srv872082844:84 , srv875693683:84 , srv876922201:84 , srv959364053:84 , srv1137693895:85 , srv1191041757:85 , srv1234083891:85 , srv1272206980:85 , srv1284672376:85 , srv1329432781:85 , srv1360916389:85 , srv1385974267:85 , srv139098792:85 , srv1406576910:85 , srv1447794946:85 , srv1476916941:85 , srv1501007099:85 , srv1582060181:85 , srv177789160:85 , srv1807640493:85 , srv1862489435:85 , srv1911872306:85 , srv1953141633:85 , srv1961997052:85 , srv2004227232:85 , srv2048707461:85 , srv205126670:85 , srv2137807933:85 , srv284045326:85 , srv424521173:85 , srv480549382:85 , srv518731295:85 , srv519399986:85 , srv539220227:85 , srv565767614:85 , srv584568139:85 , srv603413215:85 , srv643571435:85 , srv661365551:85 , srv738634588:85 , srv754162017:85 , srv815443750:85 , srv830434270:85 , srv899463815:85 , srv906258380:85 , srv991964466:85 , srv1023726577:86 , srv1072250486:86 , srv1252325856:86 , srv1709007636:86 , srv1730221160:86 , srv1828587780:86 , srv1879724488:86 , srv1976631629:86 , srv1982585649:86 , srv1993640754:86 , srv2045598307:86 , srv255177449:86 , srv401169459:86 , srv520593614:86 , srv619899731:86 , srv776574302:86 , srv865719484:86 , srv543382739:87 , srv1405991983:612 }
2023-11-02 15:55:50,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-02 15:55:50,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1137693895=4, srv1408532845=17, srv1410186028=18, srv1197373626=6, srv80345466=88, srv1498132383=21, srv59988934=73, srv1385974267=13, srv1406576910=16, srv609382280=75, srv1948149954=41, srv1272206980=9, srv876922201=95, srv1191041757=5, srv661365551=80, srv772759390=84, srv1585830226=25, srv603413215=74, srv188069612=38, srv815443750=89, srv1807640493=33, srv652982955=79, srv738634588=82, srv139098792=14, srv520593614=66, srv518731295=64, srv1730221160=29, srv906258380=97, srv1072250486=1, srv793417979=87, srv754162017=83, srv991964466=99, srv218190829=53, srv177789160=30, srv1582060181=24, srv1405991983=15, srv1993640754=46, srv1503368570=23, srv1879724488=37, srv401169459=60, srv1982585649=45, srv584568139=72, srv1284672376=10, srv1709007636=28, srv248852011=55, srv959364053=98, srv1787020744=31, srv1828587780=34, srv519399986=65, srv1911872306=39, srv875693683=94, srv2004227232=48, srv737280100=81, srv526053339=67, srv1961997052=43, srv1862489435=35, srv237101174=54, srv424521173=61, srv776574302=85, srv865719484=92, srv1639757848=26, srv1787454732=32, srv899463815=96, srv205126670=51, srv1329432781=11, srv1999429103=47, srv549149739=70, srv830434270=90, srv1663443158=27, srv2045598307=49, srv1234083891=7, srv284045326=57, srv1023726577=0, srv372702083=58, srv1476916941=20, srv1976631629=44, srv779283332=86, srv831594694=91, srv646062640=78, srv1074668308=2, srv543382739=69, srv1912636597=40, srv1501007099=22, srv1252325856=8, srv619899731=76, srv643571435=77, srv1953141633=42, srv255177449=56, srv1360916389=12, srv1447794946=19, srv2137807933=52, srv110345042=3, srv2048707461=50, srv872082844=93, srv480549382=62, srv1865003625=36, srv377319443=59, srv539220227=68, srv48626042=63, srv565767614=71} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-02 15:55:50,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-02 15:55:50,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-02 15:55:50,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-02 15:55:50,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-02 15:55:50,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-02 15:55:50,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-02 15:55:50,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-02 15:55:50,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-02 15:55:50,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-02 15:55:50,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-02 15:55:50,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-02 15:55:50,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-02 15:55:50,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-02 15:55:50,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-02 15:55:50,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-02 15:55:50,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-02 15:55:50,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-02 15:55:50,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-02 15:55:50,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-02 15:55:50,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-02 15:55:50,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-02 15:55:50,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-02 15:55:50,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-02 15:55:50,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-02 15:55:50,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-02 15:55:50,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-02 15:55:50,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-02 15:55:50,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-02 15:55:50,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-02 15:55:50,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-02 15:55:50,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-02 15:55:50,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-02 15:55:50,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-02 15:55:50,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-02 15:55:50,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-02 15:55:50,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-02 15:55:50,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-02 15:55:50,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-02 15:55:50,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-02 15:55:50,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-02 15:55:50,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-02 15:55:50,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-02 15:55:50,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-02 15:55:50,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-02 15:55:50,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-02 15:55:50,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-02 15:55:50,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-02 15:55:50,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-02 15:55:50,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-02 15:55:50,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-02 15:55:50,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-02 15:55:50,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-02 15:55:50,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-02 15:55:50,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-02 15:55:50,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-02 15:55:50,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-02 15:55:50,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-02 15:55:50,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-02 15:55:50,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-02 15:55:50,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-02 15:55:50,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-02 15:55:50,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-02 15:55:50,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-02 15:55:50,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-02 15:55:50,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-02 15:55:50,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-02 15:55:50,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-02 15:55:50,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-02 15:55:50,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-02 15:55:50,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-02 15:55:50,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-02 15:55:50,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-02 15:55:50,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-02 15:55:50,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-02 15:55:50,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-02 15:55:50,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-02 15:55:50,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-02 15:55:50,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-02 15:55:50,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-02 15:55:50,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-02 15:55:50,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-02 15:55:50,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-02 15:55:50,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-02 15:55:50,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-02 15:55:50,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-02 15:55:50,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-02 15:55:50,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-02 15:55:50,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-02 15:55:50,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-02 15:55:50,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-02 15:55:50,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-02 15:55:50,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-02 15:55:50,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-02 15:55:50,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-02 15:55:50,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-02 15:55:50,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-02 15:55:50,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-02 15:55:50,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-02 15:55:50,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-02 15:55:50,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-02 15:55:50,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-02 15:55:50,871 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0021711298242630987); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=5.833333333333334E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-02 15:55:50,871 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-02 15:55:50,871 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=1.3971762369577746E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0021711298242630987); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=5.833333333333334E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-02 15:55:51,049 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 303 ms to try 5000 different iterations.  Found a solution that moves 47 regions; Going from a computed imbalance of 1.3971762369577746E-4 to a new imbalance of 7.716858534330002E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0016412198797244363); MoveCostFunction : (multiplier=7.0, imbalance=0.005222222222222222); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-02 15:55:51,050 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1023726577:83 , srv1252325856:83 , srv2045598307:83 , srv520593614:83 , srv543382739:83 , srv646062640:83 , srv738634588:83 , srv754162017:83 , srv831594694:83 , srv875693683:83 , srv1072250486:84 , srv1074668308:84 , srv110345042:84 , srv1137693895:84 , srv1191041757:84 , srv1234083891:84 , srv1272206980:84 , srv1284672376:84 , srv1329432781:84 , srv1360916389:84 , srv1385974267:84 , srv139098792:84 , srv1406576910:84 , srv1408532845:84 , srv1410186028:84 , srv1447794946:84 , srv1476916941:84 , srv1501007099:84 , srv1503368570:84 , srv1582060181:84 , srv1639757848:84 , srv1663443158:84 , srv1709007636:84 , srv1730221160:84 , srv177789160:84 , srv1787020744:84 , srv1787454732:84 , srv1807640493:84 , srv1828587780:84 , srv1862489435:84 , srv1865003625:84 , srv1879724488:84 , srv188069612:84 , srv1911872306:84 , srv1912636597:84 , srv1948149954:84 , srv1953141633:84 , srv1961997052:84 , srv1976631629:84 , srv1982585649:84 , srv1993640754:84 , srv2004227232:84 , srv2048707461:84 , srv205126670:84 , srv2137807933:84 , srv218190829:84 , srv237101174:84 , srv248852011:84 , srv255177449:84 , srv284045326:84 , srv372702083:84 , srv377319443:84 , srv401169459:84 , srv424521173:84 , srv480549382:84 , srv518731295:84 , srv519399986:84 , srv539220227:84 , srv549149739:84 , srv565767614:84 , srv584568139:84 , srv59988934:84 , srv603413215:84 , srv619899731:84 , srv643571435:84 , srv661365551:84 , srv737280100:84 , srv772759390:84 , srv776574302:84 , srv779283332:84 , srv793417979:84 , srv80345466:84 , srv830434270:84 , srv865719484:84 , srv872082844:84 , srv876922201:84 , srv899463815:84 , srv906258380:84 , srv959364053:84 , srv1197373626:85 , srv1498132383:85 , srv1585830226:85 , srv1999429103:85 , srv48626042:85 , srv526053339:85 , srv609382280:85 , srv652982955:85 , srv815443750:85 , srv991964466:85 , srv1405991983:684 }
2023-11-02 15:55:51,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-02 15:55:51,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1137693895=4, srv1408532845=17, srv1410186028=18, srv1197373626=6, srv80345466=88, srv1498132383=21, srv59988934=73, srv1385974267=13, srv1406576910=16, srv609382280=75, srv1948149954=41, srv1272206980=9, srv876922201=95, srv1191041757=5, srv661365551=80, srv772759390=84, srv1585830226=25, srv603413215=74, srv188069612=38, srv815443750=89, srv1807640493=33, srv652982955=79, srv738634588=82, srv139098792=14, srv520593614=66, srv518731295=64, srv1730221160=29, srv906258380=97, srv1072250486=1, srv793417979=87, srv754162017=83, srv991964466=99, srv218190829=53, srv177789160=30, srv1582060181=24, srv1405991983=15, srv1993640754=46, srv1503368570=23, srv1879724488=37, srv401169459=60, srv1982585649=45, srv584568139=72, srv1284672376=10, srv1709007636=28, srv248852011=55, srv959364053=98, srv1787020744=31, srv1828587780=34, srv519399986=65, srv1911872306=39, srv875693683=94, srv2004227232=48, srv737280100=81, srv526053339=67, srv1961997052=43, srv1862489435=35, srv237101174=54, srv424521173=61, srv776574302=85, srv865719484=92, srv1639757848=26, srv1787454732=32, srv899463815=96, srv205126670=51, srv1329432781=11, srv1999429103=47, srv549149739=70, srv830434270=90, srv1663443158=27, srv2045598307=49, srv1234083891=7, srv284045326=57, srv1023726577=0, srv372702083=58, srv1476916941=20, srv1976631629=44, srv779283332=86, srv831594694=91, srv646062640=78, srv1074668308=2, srv543382739=69, srv1912636597=40, srv1501007099=22, srv1252325856=8, srv619899731=76, srv643571435=77, srv1953141633=42, srv255177449=56, srv1360916389=12, srv1447794946=19, srv2137807933=52, srv110345042=3, srv2048707461=50, srv872082844=93, srv480549382=62, srv1865003625=36, srv377319443=59, srv539220227=68, srv48626042=63, srv565767614=71} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-02 15:55:51,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-02 15:55:51,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-02 15:55:51,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-02 15:55:51,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-02 15:55:51,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-02 15:55:51,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-02 15:55:51,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-02 15:55:51,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-02 15:55:51,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-02 15:55:51,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-02 15:55:51,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-02 15:55:51,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-02 15:55:51,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-02 15:55:51,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-02 15:55:51,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-02 15:55:51,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-02 15:55:51,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-02 15:55:51,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-02 15:55:51,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-02 15:55:51,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-02 15:55:51,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-02 15:55:51,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-02 15:55:51,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-02 15:55:51,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-02 15:55:51,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-02 15:55:51,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-02 15:55:51,072 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-02 15:55:51,072 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-02 15:55:51,072 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-02 15:55:51,072 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-02 15:55:51,072 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-02 15:55:51,072 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-02 15:55:51,072 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-02 15:55:51,072 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-02 15:55:51,072 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-02 15:55:51,072 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-02 15:55:51,072 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-02 15:55:51,072 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-02 15:55:51,073 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-02 15:55:51,073 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-02 15:55:51,073 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-02 15:55:51,073 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-02 15:55:51,073 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-02 15:55:51,073 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-02 15:55:51,073 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-02 15:55:51,073 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-02 15:55:51,073 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-02 15:55:51,073 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-02 15:55:51,073 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-02 15:55:51,073 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-02 15:55:51,073 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-02 15:55:51,074 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-02 15:55:51,074 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-02 15:55:51,074 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-02 15:55:51,074 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-02 15:55:51,074 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-02 15:55:51,074 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-02 15:55:51,074 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-02 15:55:51,074 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-02 15:55:51,074 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-02 15:55:51,074 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-02 15:55:51,074 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-02 15:55:51,074 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-02 15:55:51,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-02 15:55:51,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-02 15:55:51,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-02 15:55:51,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-02 15:55:51,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-02 15:55:51,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-02 15:55:51,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-02 15:55:51,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-02 15:55:51,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-02 15:55:51,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-02 15:55:51,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-02 15:55:51,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-02 15:55:51,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-02 15:55:51,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-02 15:55:51,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-02 15:55:51,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-02 15:55:51,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-02 15:55:51,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-02 15:55:51,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-02 15:55:51,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-02 15:55:51,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-02 15:55:51,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-02 15:55:51,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-02 15:55:51,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-02 15:55:51,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-02 15:55:51,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-02 15:55:51,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-02 15:55:51,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-02 15:55:51,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-02 15:55:51,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-02 15:55:51,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-02 15:55:51,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-02 15:55:51,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-02 15:55:51,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-02 15:55:51,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-02 15:55:51,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-02 15:55:51,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-02 15:55:51,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-02 15:55:51,178 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=7.387757500312559E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0016412198797244363); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-02 15:55:51,178 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1023726577:83 , srv1252325856:83 , srv2045598307:83 , srv520593614:83 , srv543382739:83 , srv646062640:83 , srv738634588:83 , srv754162017:83 , srv831594694:83 , srv875693683:83 , srv1072250486:84 , srv1074668308:84 , srv110345042:84 , srv1137693895:84 , srv1191041757:84 , srv1234083891:84 , srv1272206980:84 , srv1284672376:84 , srv1329432781:84 , srv1360916389:84 , srv1385974267:84 , srv139098792:84 , srv1406576910:84 , srv1408532845:84 , srv1410186028:84 , srv1447794946:84 , srv1476916941:84 , srv1501007099:84 , srv1503368570:84 , srv1582060181:84 , srv1639757848:84 , srv1663443158:84 , srv1709007636:84 , srv1730221160:84 , srv177789160:84 , srv1787020744:84 , srv1787454732:84 , srv1807640493:84 , srv1828587780:84 , srv1862489435:84 , srv1865003625:84 , srv1879724488:84 , srv188069612:84 , srv1911872306:84 , srv1912636597:84 , srv1948149954:84 , srv1953141633:84 , srv1961997052:84 , srv1976631629:84 , srv1982585649:84 , srv1993640754:84 , srv2004227232:84 , srv2048707461:84 , srv205126670:84 , srv2137807933:84 , srv218190829:84 , srv237101174:84 , srv248852011:84 , srv255177449:84 , srv284045326:84 , srv372702083:84 , srv377319443:84 , srv401169459:84 , srv424521173:84 , srv480549382:84 , srv518731295:84 , srv519399986:84 , srv539220227:84 , srv549149739:84 , srv565767614:84 , srv584568139:84 , srv59988934:84 , srv603413215:84 , srv619899731:84 , srv643571435:84 , srv661365551:84 , srv737280100:84 , srv772759390:84 , srv776574302:84 , srv779283332:84 , srv793417979:84 , srv80345466:84 , srv830434270:84 , srv865719484:84 , srv872082844:84 , srv876922201:84 , srv899463815:84 , srv906258380:84 , srv959364053:84 , srv1197373626:85 , srv1498132383:85 , srv1585830226:85 , srv1999429103:85 , srv48626042:85 , srv526053339:85 , srv609382280:85 , srv652982955:85 , srv815443750:85 , srv991964466:85 , srv1405991983:684 }
2023-11-02 15:55:51,199 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=501 (was 501), ProcessCount=179 (was 179), AvailableMemoryMB=1651 (was 2200)
2023-11-02 15:55:51,207 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=501, ProcessCount=179, AvailableMemoryMB=1651
2023-11-02 15:55:51,207 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-02 15:55:51,208 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-11-02 15:55:51,219 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1009812178:84 , srv1012287567:84 , srv1118019178:84 , srv1148812254:84 , srv1215610235:84 , srv1222032547:84 , srv1224909943:84 , srv126914167:84 , srv1280615942:84 , srv1432883285:84 , srv1530706761:84 , srv153252117:84 , srv1631821808:84 , srv1661898753:84 , srv1732888651:84 , srv1787881569:84 , srv2051098565:84 , srv2110815603:84 , srv270197400:84 , srv394531462:84 , srv396066642:84 , srv513438202:84 , srv566993735:84 , srv678144382:84 , srv782548505:84 , srv842764247:84 , srv879977616:84 , srv945044489:84 , srv95028118:84 , srv50115363:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-11-02 15:55:51,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-02 15:55:51,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1224909943=6, srv2110815603=17, srv1280615942=8, srv126914167=7, srv1661898753=13, srv153252117=11, srv1012287567=1, srv1118019178=2, srv1148812254=3, srv1631821808=12, srv1215610235=4, srv1787881569=15, srv1732888651=14, srv1009812178=0, srv678144382=24, srv50115363=21, srv879977616=27, srv842764247=26, srv945044489=28, srv394531462=19, srv1222032547=5, srv782548505=25, srv1432883285=9, srv2051098565=16, srv396066642=20, srv1530706761=10, srv95028118=29, srv270197400=18, srv513438202=22, srv566993735=23} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-02 15:55:51,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-02 15:55:51,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-02 15:55:51,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-02 15:55:51,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-02 15:55:51,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-02 15:55:51,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-02 15:55:51,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-02 15:55:51,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-02 15:55:51,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-02 15:55:51,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-02 15:55:51,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-02 15:55:51,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-02 15:55:51,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-02 15:55:51,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-02 15:55:51,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-02 15:55:51,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-02 15:55:51,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-02 15:55:51,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-02 15:55:51,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-02 15:55:51,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-02 15:55:51,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-02 15:55:51,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-02 15:55:51,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-02 15:55:51,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-02 15:55:51,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-02 15:55:51,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-02 15:55:51,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-02 15:55:51,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-02 15:55:51,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-02 15:55:51,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-02 15:55:51,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-11-02 15:55:51,246 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.11633273973542, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-02 15:55:51,246 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912559603928573, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.11633273973542, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-11-02 15:57:51,235 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 8682648 different iterations.  Found a solution that moves 1842 regions; Going from a computed imbalance of 0.9912559603928573 to a new imbalance of 4.2993198912066E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6822222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-02 15:57:51,242 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1009812178:90 , srv1012287567:90 , srv1118019178:90 , srv1148812254:90 , srv1215610235:90 , srv1222032547:90 , srv1224909943:90 , srv126914167:90 , srv1280615942:90 , srv1432883285:90 , srv1530706761:90 , srv153252117:90 , srv1631821808:90 , srv1661898753:90 , srv1732888651:90 , srv1787881569:90 , srv2051098565:90 , srv2110815603:90 , srv270197400:90 , srv394531462:90 , srv396066642:90 , srv50115363:90 , srv513438202:90 , srv566993735:90 , srv678144382:90 , srv782548505:90 , srv842764247:90 , srv879977616:90 , srv945044489:90 , srv95028118:90 }
2023-11-02 15:57:51,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-02 15:57:51,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1224909943=6, srv2110815603=17, srv1280615942=8, srv126914167=7, srv1661898753=13, srv153252117=11, srv1012287567=1, srv1118019178=2, srv1148812254=3, srv1631821808=12, srv1215610235=4, srv1787881569=15, srv1732888651=14, srv1009812178=0, srv678144382=24, srv50115363=21, srv879977616=27, srv842764247=26, srv945044489=28, srv394531462=19, srv1222032547=5, srv782548505=25, srv1432883285=9, srv2051098565=16, srv396066642=20, srv1530706761=10, srv95028118=29, srv270197400=18, srv513438202=22, srv566993735=23} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-02 15:57:51,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-02 15:57:51,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-02 15:57:51,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-02 15:57:51,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-02 15:57:51,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-02 15:57:51,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-02 15:57:51,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-02 15:57:51,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-02 15:57:51,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-02 15:57:51,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-02 15:57:51,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-02 15:57:51,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-02 15:57:51,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-02 15:57:51,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-02 15:57:51,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-02 15:57:51,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-02 15:57:51,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-02 15:57:51,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-02 15:57:51,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-02 15:57:51,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-02 15:57:51,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-02 15:57:51,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-02 15:57:51,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-02 15:57:51,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-02 15:57:51,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-02 15:57:51,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-02 15:57:51,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-02 15:57:51,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-02 15:57:51,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-02 15:57:51,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-02 15:57:51,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-11-02 15:57:51,262 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-02 15:57:51,262 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1009812178:90 , srv1012287567:90 , srv1118019178:90 , srv1148812254:90 , srv1215610235:90 , srv1222032547:90 , srv1224909943:90 , srv126914167:90 , srv1280615942:90 , srv1432883285:90 , srv1530706761:90 , srv153252117:90 , srv1631821808:90 , srv1661898753:90 , srv1732888651:90 , srv1787881569:90 , srv2051098565:90 , srv2110815603:90 , srv270197400:90 , srv394531462:90 , srv396066642:90 , srv50115363:90 , srv513438202:90 , srv566993735:90 , srv678144382:90 , srv782548505:90 , srv842764247:90 , srv879977616:90 , srv945044489:90 , srv95028118:90 }
2023-11-02 15:57:51,284 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=590 (was 501) - SystemLoadAverage LEAK? -, ProcessCount=173 (was 179), AvailableMemoryMB=3157 (was 1651) - AvailableMemoryMB LEAK? -