2024-02-04 06:53:09,382 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2024-02-04 06:53:09,699 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2024-02-04 06:53:09,808 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2024-02-04 06:53:09,878 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-02-04 06:53:09,885 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-02-04 06:53:09,898 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=273, ProcessCount=169, AvailableMemoryMB=3173
2024-02-04 06:53:09,900 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-02-04 06:53:09,901 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-02-04 06:53:10,035 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1018024614:84 , srv1036499633:84 , srv1043894469:84 , srv1076041235:84 , srv1077680278:84 , srv1081706253:84 , srv1102971773:84 , srv1111795145:84 , srv1136466995:84 , srv1156630340:84 , srv1161686194:84 , srv1170333804:84 , srv1194489305:84 , srv123891052:84 , srv1242045651:84 , srv1252785698:84 , srv1320695810:84 , srv1340458023:84 , srv1349824585:84 , srv1371921100:84 , srv1416102381:84 , srv141905368:84 , srv1421441163:84 , srv143782906:84 , srv1502180515:84 , srv1502462805:84 , srv1521010423:84 , srv1537889511:84 , srv1551478433:84 , srv1577844259:84 , srv1583996552:84 , srv1594557012:84 , srv1601958875:84 , srv1621519136:84 , srv1638874755:84 , srv1653414266:84 , srv16536197:84 , srv1654183980:84 , srv1674085559:84 , srv172751691:84 , srv1753669075:84 , srv1764086977:84 , srv1777969007:84 , srv1786928863:84 , srv1808020694:84 , srv1874390986:84 , srv1879549038:84 , srv1896572343:84 , srv1949395799:84 , srv1955649062:84 , srv1966307453:84 , srv1982178250:84 , srv1989849870:84 , srv2031776324:84 , srv2035105130:84 , srv2044207070:84 , srv2046508418:84 , srv2074221824:84 , srv2078328648:84 , srv2106710287:84 , srv215175938:84 , srv244703426:84 , srv251823120:84 , srv303277950:84 , srv353981903:84 , srv368460176:84 , srv372521837:84 , srv403463459:84 , srv419377234:84 , srv424847848:84 , srv433518298:84 , srv434622083:84 , srv446068126:84 , srv515493474:84 , srv535801785:84 , srv54286418:84 , srv546874368:84 , srv598977505:84 , srv602337233:84 , srv618720744:84 , srv624063149:84 , srv636153171:84 , srv663368223:84 , srv675392462:84 , srv677217386:84 , srv677338002:84 , srv690159073:84 , srv701561508:84 , srv703023535:84 , srv715291162:84 , srv774021411:84 , srv7749794:84 , srv804780671:84 , srv818082709:84 , srv886080826:84 , srv892347169:84 , srv920084929:84 , srv988921495:84 , srv995151903:84 , srv1961707590:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2024-02-04 06:53:10,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-04 06:53:10,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1416102381=20, srv1194489305=12, srv353981903=65, srv303277950=64, srv1036499633=1, srv1018024614=0, srv818082709=94, srv515493474=74, srv123891052=13, srv703023535=89, srv1638874755=34, srv1786928863=43, srv141905368=21, srv1349824585=18, srv1076041235=3, srv1577844259=29, srv215175938=61, srv434622083=72, srv1077680278=4, srv1242045651=14, srv251823120=63, srv1949395799=48, srv618720744=80, srv368460176=66, srv988921495=98, srv1654183980=37, srv701561508=88, srv1537889511=27, srv1621519136=33, srv2078328648=59, srv675392462=84, srv172751691=39, srv372521837=67, srv715291162=90, srv1102971773=6, srv1874390986=45, srv1594557012=31, srv2106710287=60, srv1777969007=42, srv1989849870=53, srv403463459=68, srv1081706253=5, srv602337233=79, srv1896572343=47, srv1521010423=26, srv1252785698=15, srv677217386=85, srv143782906=23, srv1551478433=28, srv1961707590=50, srv2035105130=55, srv920084929=97, srv2031776324=54, srv2044207070=56, srv546874368=77, srv446068126=73, srv690159073=87, srv804780671=93, srv1421441163=22, srv995151903=99, srv2074221824=58, srv1753669075=40, srv424847848=70, srv244703426=62, srv886080826=95, srv1982178250=52, srv1502462805=25, srv1340458023=17, srv1966307453=51, srv1808020694=44, srv7749794=92, srv1111795145=7, srv2046508418=57, srv1955649062=49, srv1161686194=10, srv1320695810=16, srv1371921100=19, srv419377234=69, srv774021411=91, srv1653414266=35, srv663368223=83, srv1674085559=38, srv1136466995=8, srv54286418=76, srv535801785=75, srv1601958875=32, srv636153171=82, srv624063149=81, srv677338002=86, srv16536197=36, srv1583996552=30, srv433518298=71, srv1879549038=46, srv598977505=78, srv1043894469=2, srv1156630340=9, srv892347169=96, srv1502180515=24, srv1764086977=41, srv1170333804=11} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-04 06:53:10,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-04 06:53:10,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-04 06:53:10,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-04 06:53:10,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-04 06:53:10,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-04 06:53:10,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-04 06:53:10,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-04 06:53:10,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-04 06:53:10,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-04 06:53:10,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-04 06:53:10,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-04 06:53:10,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-04 06:53:10,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-04 06:53:10,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-04 06:53:10,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-04 06:53:10,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-04 06:53:10,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-04 06:53:10,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-04 06:53:10,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-04 06:53:10,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-04 06:53:10,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-04 06:53:10,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-04 06:53:10,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-04 06:53:10,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-04 06:53:10,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-04 06:53:10,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-04 06:53:10,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-04 06:53:10,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-04 06:53:10,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-04 06:53:10,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-04 06:53:10,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-02-04 06:53:10,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-02-04 06:53:10,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-02-04 06:53:10,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-02-04 06:53:10,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-02-04 06:53:10,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-02-04 06:53:10,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-02-04 06:53:10,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-02-04 06:53:10,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-02-04 06:53:10,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-02-04 06:53:10,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-02-04 06:53:10,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-02-04 06:53:10,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-02-04 06:53:10,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-02-04 06:53:10,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-02-04 06:53:10,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-02-04 06:53:10,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-02-04 06:53:10,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-02-04 06:53:10,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-02-04 06:53:10,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-02-04 06:53:10,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-02-04 06:53:10,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-02-04 06:53:10,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-02-04 06:53:10,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-02-04 06:53:10,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-02-04 06:53:10,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-02-04 06:53:10,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-02-04 06:53:10,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-02-04 06:53:10,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-02-04 06:53:10,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-02-04 06:53:10,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-02-04 06:53:10,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-02-04 06:53:10,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-02-04 06:53:10,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-02-04 06:53:10,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-02-04 06:53:10,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-02-04 06:53:10,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-02-04 06:53:10,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-02-04 06:53:10,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-02-04 06:53:10,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-02-04 06:53:10,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-02-04 06:53:10,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-02-04 06:53:10,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-02-04 06:53:10,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-02-04 06:53:10,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-02-04 06:53:10,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-02-04 06:53:10,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-02-04 06:53:10,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-02-04 06:53:10,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-02-04 06:53:10,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-02-04 06:53:10,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-02-04 06:53:10,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-02-04 06:53:10,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-02-04 06:53:10,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-02-04 06:53:10,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-02-04 06:53:10,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-02-04 06:53:10,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-02-04 06:53:10,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-02-04 06:53:10,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-02-04 06:53:10,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-02-04 06:53:10,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-02-04 06:53:10,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-02-04 06:53:10,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-02-04 06:53:10,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-02-04 06:53:10,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-02-04 06:53:10,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-02-04 06:53:10,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-02-04 06:53:10,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-02-04 06:53:10,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-02-04 06:53:10,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-02-04 06:53:10,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-02-04 06:53:10,399 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-04 06:53:10,399 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-02-04 06:53:10,399 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-02-04 06:53:10,762 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 548 ms to try 5000 different iterations.  Found a solution that moves 4687 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1820591619394063. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.041062397201067566, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05561007425949929, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5207777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.041062397201067566, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17741666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24275, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-04 06:53:10,778 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1018024614:80 , srv1077680278:81 , srv433518298:81 , srv1194489305:82 , srv1320695810:82 , srv1537889511:82 , srv535801785:82 , srv1601958875:83 , srv701561508:83 , srv920084929:83 , srv141905368:84 , srv1577844259:84 , srv1621519136:84 , srv2078328648:84 , srv2106710287:84 , srv244703426:84 , srv251823120:84 , srv372521837:84 , srv703023535:84 , srv1340458023:85 , srv1416102381:85 , srv143782906:85 , srv1583996552:85 , srv16536197:85 , srv1753669075:85 , srv1786928863:85 , srv1879549038:85 , srv1896572343:85 , srv1949395799:85 , srv1955649062:85 , srv1989849870:85 , srv353981903:85 , srv636153171:85 , srv1036499633:86 , srv1043894469:86 , srv1076041235:86 , srv1102971773:86 , srv1111795145:86 , srv1161686194:86 , srv123891052:86 , srv1349824585:86 , srv1371921100:86 , srv1421441163:86 , srv1502462805:86 , srv1551478433:86 , srv1764086977:86 , srv2035105130:86 , srv368460176:86 , srv434622083:86 , srv54286418:86 , srv546874368:86 , srv602337233:86 , srv618720744:86 , srv624063149:86 , srv715291162:86 , srv804780671:86 , srv988921495:86 , srv1136466995:87 , srv1156630340:87 , srv1170333804:87 , srv1502180515:87 , srv1638874755:87 , srv2031776324:87 , srv215175938:87 , srv303277950:87 , srv403463459:87 , srv515493474:87 , srv675392462:87 , srv677217386:87 , srv886080826:87 , srv995151903:87 , srv1081706253:88 , srv1594557012:88 , srv1777969007:88 , srv1808020694:88 , srv1874390986:88 , srv1982178250:88 , srv2044207070:88 , srv2046508418:88 , srv424847848:88 , srv598977505:88 , srv690159073:88 , srv774021411:88 , srv7749794:88 , srv818082709:88 , srv1252785698:89 , srv1653414266:89 , srv1654183980:89 , srv1966307453:89 , srv446068126:89 , srv663368223:89 , srv892347169:89 , srv1521010423:90 , srv172751691:90 , srv2074221824:90 , srv1674085559:91 , srv1242045651:92 , srv677338002:92 , srv419377234:98 , srv1961707590:455 }
2024-02-04 06:53:10,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-04 06:53:10,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1416102381=20, srv1194489305=12, srv353981903=65, srv303277950=64, srv1036499633=1, srv1018024614=0, srv818082709=94, srv515493474=74, srv123891052=13, srv703023535=89, srv1638874755=34, srv1786928863=43, srv141905368=21, srv1349824585=18, srv1076041235=3, srv1577844259=29, srv215175938=61, srv434622083=72, srv1077680278=4, srv1242045651=14, srv251823120=63, srv1949395799=48, srv618720744=80, srv368460176=66, srv988921495=98, srv1654183980=37, srv701561508=88, srv1537889511=27, srv1621519136=33, srv2078328648=59, srv675392462=84, srv172751691=39, srv372521837=67, srv715291162=90, srv1102971773=6, srv1874390986=45, srv1594557012=31, srv2106710287=60, srv1777969007=42, srv1989849870=53, srv403463459=68, srv1081706253=5, srv602337233=79, srv1896572343=47, srv1521010423=26, srv1252785698=15, srv677217386=85, srv143782906=23, srv1551478433=28, srv1961707590=50, srv2035105130=55, srv920084929=97, srv2031776324=54, srv2044207070=56, srv546874368=77, srv446068126=73, srv690159073=87, srv804780671=93, srv1421441163=22, srv995151903=99, srv2074221824=58, srv1753669075=40, srv424847848=70, srv244703426=62, srv886080826=95, srv1982178250=52, srv1502462805=25, srv1340458023=17, srv1966307453=51, srv1808020694=44, srv7749794=92, srv1111795145=7, srv2046508418=57, srv1955649062=49, srv1161686194=10, srv1320695810=16, srv1371921100=19, srv419377234=69, srv774021411=91, srv1653414266=35, srv663368223=83, srv1674085559=38, srv1136466995=8, srv54286418=76, srv535801785=75, srv1601958875=32, srv636153171=82, srv624063149=81, srv677338002=86, srv16536197=36, srv1583996552=30, srv433518298=71, srv1879549038=46, srv598977505=78, srv1043894469=2, srv1156630340=9, srv892347169=96, srv1502180515=24, srv1764086977=41, srv1170333804=11} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-04 06:53:10,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-04 06:53:10,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-04 06:53:10,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-04 06:53:10,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-04 06:53:10,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-04 06:53:10,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-04 06:53:10,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-04 06:53:10,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-04 06:53:10,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-04 06:53:10,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-04 06:53:10,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-04 06:53:10,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-04 06:53:10,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-04 06:53:10,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-04 06:53:10,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-04 06:53:10,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-04 06:53:10,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-04 06:53:10,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-04 06:53:10,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-04 06:53:10,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-04 06:53:10,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-04 06:53:10,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-04 06:53:10,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-04 06:53:10,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-04 06:53:10,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-04 06:53:10,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-04 06:53:10,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-04 06:53:10,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-04 06:53:10,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-04 06:53:10,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-04 06:53:10,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-02-04 06:53:10,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-02-04 06:53:10,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-02-04 06:53:10,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-02-04 06:53:10,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-02-04 06:53:10,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-02-04 06:53:10,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-02-04 06:53:10,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-02-04 06:53:10,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-02-04 06:53:10,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-02-04 06:53:10,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-02-04 06:53:10,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-02-04 06:53:10,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-02-04 06:53:10,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-02-04 06:53:10,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-02-04 06:53:10,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-02-04 06:53:10,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-02-04 06:53:10,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-02-04 06:53:10,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-02-04 06:53:10,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-02-04 06:53:10,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-02-04 06:53:10,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-02-04 06:53:10,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-02-04 06:53:10,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-02-04 06:53:10,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-02-04 06:53:10,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-02-04 06:53:10,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-02-04 06:53:10,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-02-04 06:53:10,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-02-04 06:53:10,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-02-04 06:53:10,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-02-04 06:53:10,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-02-04 06:53:10,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-02-04 06:53:10,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-02-04 06:53:10,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-02-04 06:53:10,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-02-04 06:53:10,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-02-04 06:53:10,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-02-04 06:53:10,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-02-04 06:53:10,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-02-04 06:53:10,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-02-04 06:53:10,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-02-04 06:53:10,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-02-04 06:53:10,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-02-04 06:53:10,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-02-04 06:53:10,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-02-04 06:53:10,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-02-04 06:53:10,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-02-04 06:53:10,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-02-04 06:53:10,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-02-04 06:53:10,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-02-04 06:53:10,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-02-04 06:53:10,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-02-04 06:53:10,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-02-04 06:53:10,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-02-04 06:53:10,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-02-04 06:53:10,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-02-04 06:53:10,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-02-04 06:53:10,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-02-04 06:53:10,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-02-04 06:53:10,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-02-04 06:53:10,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-02-04 06:53:10,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-02-04 06:53:10,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-02-04 06:53:10,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-02-04 06:53:10,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-02-04 06:53:10,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-02-04 06:53:10,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-02-04 06:53:10,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-02-04 06:53:10,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-02-04 06:53:10,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-02-04 06:53:10,978 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.041062397201067566, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05561007425949929, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.041062397201067566, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17741666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24275, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-04 06:53:10,979 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-02-04 06:53:10,979 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18202634286395014, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.041062397201067566, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05561007425949929, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.041062397201067566, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17741666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24275, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-02-04 06:53:11,210 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 346 ms to try 5000 different iterations.  Found a solution that moves 2619 regions; Going from a computed imbalance of 0.18202634286395014 to a new imbalance of 0.013802791525794758. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022623206062585232); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03639785146777565, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.291, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022623206062585232); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010666666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043416666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-04 06:53:11,221 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv419377234:75 , srv1242045651:78 , srv172751691:80 , srv1252785698:81 , srv1521010423:81 , srv1674085559:81 , srv1653414266:82 , srv1966307453:82 , srv2044207070:82 , srv1081706253:83 , srv1874390986:83 , srv1982178250:83 , srv2074221824:83 , srv546874368:83 , srv663368223:83 , srv677338002:83 , srv774021411:83 , srv892347169:83 , srv1594557012:84 , srv1654183980:84 , srv1808020694:84 , srv403463459:84 , srv446068126:84 , srv598977505:84 , srv675392462:84 , srv677217386:84 , srv818082709:84 , srv995151903:84 , srv1043894469:85 , srv1076041235:85 , srv1102971773:85 , srv1111795145:85 , srv1170333804:85 , srv1416102381:85 , srv1502462805:85 , srv1551478433:85 , srv1638874755:85 , srv1777969007:85 , srv1786928863:85 , srv1949395799:85 , srv1955649062:85 , srv2031776324:85 , srv215175938:85 , srv353981903:85 , srv368460176:85 , srv424847848:85 , srv515493474:85 , srv54286418:85 , srv602337233:85 , srv618720744:85 , srv636153171:85 , srv690159073:85 , srv7749794:85 , srv1136466995:86 , srv1156630340:86 , srv1161686194:86 , srv123891052:86 , srv1371921100:86 , srv1421441163:86 , srv1753669075:86 , srv1764086977:86 , srv2046508418:86 , srv2106710287:86 , srv244703426:86 , srv715291162:86 , srv1036499633:87 , srv1194489305:87 , srv1340458023:87 , srv1349824585:87 , srv1502180515:87 , srv1577844259:87 , srv1583996552:87 , srv16536197:87 , srv2078328648:87 , srv804780671:87 , srv988921495:87 , srv1320695810:88 , srv141905368:88 , srv143782906:88 , srv1896572343:88 , srv1989849870:88 , srv2035105130:88 , srv251823120:88 , srv434622083:88 , srv920084929:88 , srv1621519136:89 , srv1879549038:89 , srv624063149:89 , srv701561508:89 , srv703023535:89 , srv886080826:89 , srv1018024614:90 , srv303277950:90 , srv372521837:90 , srv433518298:90 , srv1601958875:92 , srv1077680278:93 , srv1537889511:93 , srv535801785:93 , srv1961707590:520 }
2024-02-04 06:53:11,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-04 06:53:11,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1416102381=20, srv1194489305=12, srv353981903=65, srv303277950=64, srv1036499633=1, srv1018024614=0, srv818082709=94, srv515493474=74, srv123891052=13, srv703023535=89, srv1638874755=34, srv1786928863=43, srv141905368=21, srv1349824585=18, srv1076041235=3, srv1577844259=29, srv215175938=61, srv434622083=72, srv1077680278=4, srv1242045651=14, srv251823120=63, srv1949395799=48, srv618720744=80, srv368460176=66, srv988921495=98, srv1654183980=37, srv701561508=88, srv1537889511=27, srv1621519136=33, srv2078328648=59, srv675392462=84, srv172751691=39, srv372521837=67, srv715291162=90, srv1102971773=6, srv1874390986=45, srv1594557012=31, srv2106710287=60, srv1777969007=42, srv1989849870=53, srv403463459=68, srv1081706253=5, srv602337233=79, srv1896572343=47, srv1521010423=26, srv1252785698=15, srv677217386=85, srv143782906=23, srv1551478433=28, srv1961707590=50, srv2035105130=55, srv920084929=97, srv2031776324=54, srv2044207070=56, srv546874368=77, srv446068126=73, srv690159073=87, srv804780671=93, srv1421441163=22, srv995151903=99, srv2074221824=58, srv1753669075=40, srv424847848=70, srv244703426=62, srv886080826=95, srv1982178250=52, srv1502462805=25, srv1340458023=17, srv1966307453=51, srv1808020694=44, srv7749794=92, srv1111795145=7, srv2046508418=57, srv1955649062=49, srv1161686194=10, srv1320695810=16, srv1371921100=19, srv419377234=69, srv774021411=91, srv1653414266=35, srv663368223=83, srv1674085559=38, srv1136466995=8, srv54286418=76, srv535801785=75, srv1601958875=32, srv636153171=82, srv624063149=81, srv677338002=86, srv16536197=36, srv1583996552=30, srv433518298=71, srv1879549038=46, srv598977505=78, srv1043894469=2, srv1156630340=9, srv892347169=96, srv1502180515=24, srv1764086977=41, srv1170333804=11} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-04 06:53:11,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-04 06:53:11,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-04 06:53:11,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-04 06:53:11,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-04 06:53:11,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-04 06:53:11,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-04 06:53:11,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-04 06:53:11,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-04 06:53:11,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-04 06:53:11,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-04 06:53:11,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-04 06:53:11,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-04 06:53:11,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-04 06:53:11,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-04 06:53:11,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-04 06:53:11,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-04 06:53:11,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-04 06:53:11,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-04 06:53:11,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-04 06:53:11,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-04 06:53:11,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-04 06:53:11,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-04 06:53:11,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-04 06:53:11,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-04 06:53:11,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-04 06:53:11,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-04 06:53:11,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-04 06:53:11,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-04 06:53:11,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-04 06:53:11,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-04 06:53:11,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-02-04 06:53:11,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-02-04 06:53:11,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-02-04 06:53:11,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-02-04 06:53:11,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-02-04 06:53:11,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-02-04 06:53:11,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-02-04 06:53:11,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-02-04 06:53:11,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-02-04 06:53:11,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-02-04 06:53:11,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-02-04 06:53:11,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-02-04 06:53:11,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-02-04 06:53:11,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-02-04 06:53:11,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-02-04 06:53:11,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-02-04 06:53:11,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-02-04 06:53:11,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-02-04 06:53:11,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-02-04 06:53:11,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-02-04 06:53:11,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-02-04 06:53:11,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-02-04 06:53:11,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-02-04 06:53:11,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-02-04 06:53:11,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-02-04 06:53:11,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-02-04 06:53:11,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-02-04 06:53:11,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-02-04 06:53:11,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-02-04 06:53:11,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-02-04 06:53:11,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-02-04 06:53:11,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-02-04 06:53:11,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-02-04 06:53:11,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-02-04 06:53:11,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-02-04 06:53:11,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-02-04 06:53:11,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-02-04 06:53:11,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-02-04 06:53:11,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-02-04 06:53:11,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-02-04 06:53:11,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-02-04 06:53:11,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-02-04 06:53:11,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-02-04 06:53:11,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-02-04 06:53:11,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-02-04 06:53:11,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-02-04 06:53:11,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-02-04 06:53:11,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-02-04 06:53:11,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-02-04 06:53:11,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-02-04 06:53:11,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-02-04 06:53:11,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-02-04 06:53:11,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-02-04 06:53:11,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-02-04 06:53:11,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-02-04 06:53:11,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-02-04 06:53:11,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-02-04 06:53:11,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-02-04 06:53:11,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-02-04 06:53:11,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-02-04 06:53:11,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-02-04 06:53:11,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-02-04 06:53:11,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-02-04 06:53:11,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-02-04 06:53:11,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-02-04 06:53:11,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-02-04 06:53:11,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-02-04 06:53:11,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-02-04 06:53:11,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-02-04 06:53:11,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-02-04 06:53:11,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-02-04 06:53:11,413 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022623206062585232); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03639785146777565, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022623206062585232); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010666666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043416666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-04 06:53:11,413 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-02-04 06:53:11,413 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.013784452895835362, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022623206062585232); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03639785146777565, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022623206062585232); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010666666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043416666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-02-04 06:53:11,654 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 370 ms to try 5000 different iterations.  Found a solution that moves 1013 regions; Going from a computed imbalance of 0.013784452895835362 to a new imbalance of 4.6126139241140055E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005759446011241319); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009732678359956285); MoveCostFunction : (multiplier=7.0, imbalance=0.11255555555555556, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005759446011241319); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00425); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-04 06:53:11,656 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv303277950:80 , srv886080826:81 , srv677338002:82 , srv1077680278:83 , srv1537889511:83 , srv1601958875:83 , srv1879549038:83 , srv535801785:83 , srv624063149:83 , srv1136466995:84 , srv143782906:84 , srv1502180515:84 , srv1896572343:84 , srv2035105130:84 , srv2046508418:84 , srv372521837:84 , srv434622083:84 , srv7749794:84 , srv1161686194:85 , srv123891052:85 , srv1349824585:85 , srv1371921100:85 , srv141905368:85 , srv1421441163:85 , srv1583996552:85 , srv1594557012:85 , srv1621519136:85 , srv16536197:85 , srv1654183980:85 , srv1674085559:85 , srv1753669075:85 , srv1764086977:85 , srv1777969007:85 , srv1808020694:85 , srv1989849870:85 , srv2031776324:85 , srv2074221824:85 , srv215175938:85 , srv419377234:85 , srv424847848:85 , srv446068126:85 , srv515493474:85 , srv598977505:85 , srv690159073:85 , srv703023535:85 , srv715291162:85 , srv804780671:85 , srv892347169:85 , srv988921495:85 , srv1036499633:86 , srv1081706253:86 , srv1102971773:86 , srv1111795145:86 , srv1156630340:86 , srv1170333804:86 , srv1340458023:86 , srv1521010423:86 , srv1551478433:86 , srv1638874755:86 , srv1653414266:86 , srv1966307453:86 , srv1982178250:86 , srv2106710287:86 , srv251823120:86 , srv368460176:86 , srv433518298:86 , srv602337233:86 , srv618720744:86 , srv663368223:86 , srv675392462:86 , srv701561508:86 , srv774021411:86 , srv818082709:86 , srv920084929:86 , srv1043894469:87 , srv1076041235:87 , srv1252785698:87 , srv1320695810:87 , srv1416102381:87 , srv1502462805:87 , srv1577844259:87 , srv172751691:87 , srv1874390986:87 , srv1949395799:87 , srv2044207070:87 , srv2078328648:87 , srv244703426:87 , srv403463459:87 , srv54286418:87 , srv677217386:87 , srv995151903:87 , srv1018024614:88 , srv1242045651:88 , srv1786928863:88 , srv1955649062:88 , srv353981903:88 , srv546874368:88 , srv636153171:88 , srv1194489305:89 , srv1961707590:534 }
2024-02-04 06:53:11,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-04 06:53:11,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1416102381=20, srv1194489305=12, srv353981903=65, srv303277950=64, srv1036499633=1, srv1018024614=0, srv818082709=94, srv515493474=74, srv123891052=13, srv703023535=89, srv1638874755=34, srv1786928863=43, srv141905368=21, srv1349824585=18, srv1076041235=3, srv1577844259=29, srv215175938=61, srv434622083=72, srv1077680278=4, srv1242045651=14, srv251823120=63, srv1949395799=48, srv618720744=80, srv368460176=66, srv988921495=98, srv1654183980=37, srv701561508=88, srv1537889511=27, srv1621519136=33, srv2078328648=59, srv675392462=84, srv172751691=39, srv372521837=67, srv715291162=90, srv1102971773=6, srv1874390986=45, srv1594557012=31, srv2106710287=60, srv1777969007=42, srv1989849870=53, srv403463459=68, srv1081706253=5, srv602337233=79, srv1896572343=47, srv1521010423=26, srv1252785698=15, srv677217386=85, srv143782906=23, srv1551478433=28, srv1961707590=50, srv2035105130=55, srv920084929=97, srv2031776324=54, srv2044207070=56, srv546874368=77, srv446068126=73, srv690159073=87, srv804780671=93, srv1421441163=22, srv995151903=99, srv2074221824=58, srv1753669075=40, srv424847848=70, srv244703426=62, srv886080826=95, srv1982178250=52, srv1502462805=25, srv1340458023=17, srv1966307453=51, srv1808020694=44, srv7749794=92, srv1111795145=7, srv2046508418=57, srv1955649062=49, srv1161686194=10, srv1320695810=16, srv1371921100=19, srv419377234=69, srv774021411=91, srv1653414266=35, srv663368223=83, srv1674085559=38, srv1136466995=8, srv54286418=76, srv535801785=75, srv1601958875=32, srv636153171=82, srv624063149=81, srv677338002=86, srv16536197=36, srv1583996552=30, srv433518298=71, srv1879549038=46, srv598977505=78, srv1043894469=2, srv1156630340=9, srv892347169=96, srv1502180515=24, srv1764086977=41, srv1170333804=11} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-04 06:53:11,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-04 06:53:11,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-04 06:53:11,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-04 06:53:11,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-04 06:53:11,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-04 06:53:11,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-04 06:53:11,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-04 06:53:11,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-04 06:53:11,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-04 06:53:11,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-04 06:53:11,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-04 06:53:11,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-04 06:53:11,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-04 06:53:11,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-04 06:53:11,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-04 06:53:11,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-04 06:53:11,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-04 06:53:11,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-04 06:53:11,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-04 06:53:11,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-04 06:53:11,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-04 06:53:11,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-04 06:53:11,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-04 06:53:11,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-04 06:53:11,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-04 06:53:11,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-04 06:53:11,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-04 06:53:11,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-04 06:53:11,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-04 06:53:11,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-04 06:53:11,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-02-04 06:53:11,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-02-04 06:53:11,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-02-04 06:53:11,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-02-04 06:53:11,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-02-04 06:53:11,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-02-04 06:53:11,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-02-04 06:53:11,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-02-04 06:53:11,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-02-04 06:53:11,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-02-04 06:53:11,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-02-04 06:53:11,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-02-04 06:53:11,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-02-04 06:53:11,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-02-04 06:53:11,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-02-04 06:53:11,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-02-04 06:53:11,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-02-04 06:53:11,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-02-04 06:53:11,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-02-04 06:53:11,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-02-04 06:53:11,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-02-04 06:53:11,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-02-04 06:53:11,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-02-04 06:53:11,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-02-04 06:53:11,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-02-04 06:53:11,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-02-04 06:53:11,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-02-04 06:53:11,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-02-04 06:53:11,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-02-04 06:53:11,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-02-04 06:53:11,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-02-04 06:53:11,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-02-04 06:53:11,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-02-04 06:53:11,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-02-04 06:53:11,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-02-04 06:53:11,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-02-04 06:53:11,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-02-04 06:53:11,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-02-04 06:53:11,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-02-04 06:53:11,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-02-04 06:53:11,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-02-04 06:53:11,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-02-04 06:53:11,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-02-04 06:53:11,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-02-04 06:53:11,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-02-04 06:53:11,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-02-04 06:53:11,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-02-04 06:53:11,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-02-04 06:53:11,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-02-04 06:53:11,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-02-04 06:53:11,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-02-04 06:53:11,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-02-04 06:53:11,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-02-04 06:53:11,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-02-04 06:53:11,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-02-04 06:53:11,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-02-04 06:53:11,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-02-04 06:53:11,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-02-04 06:53:11,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-02-04 06:53:11,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-02-04 06:53:11,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-02-04 06:53:11,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-02-04 06:53:11,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-02-04 06:53:11,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-02-04 06:53:11,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-02-04 06:53:11,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-02-04 06:53:11,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-02-04 06:53:11,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-02-04 06:53:11,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-02-04 06:53:11,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-02-04 06:53:11,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-02-04 06:53:11,811 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005759446011241319); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009732678359956285); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005759446011241319); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00425); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-04 06:53:11,811 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-02-04 06:53:11,811 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=4.541682148058757E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005759446011241319); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009732678359956285); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005759446011241319); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00425); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-02-04 06:53:12,186 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 480 ms to try 5000 different iterations.  Found a solution that moves 267 regions; Going from a computed imbalance of 4.541682148058757E-4 to a new imbalance of 2.2109564685176495E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.868392354571063E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0024158102511315956); MoveCostFunction : (multiplier=7.0, imbalance=0.029666666666666668, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.868392354571063E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-04 06:53:12,187 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1043894469:83 , srv1156630340:83 , srv1018024614:84 , srv1036499633:84 , srv1076041235:84 , srv1077680278:84 , srv1170333804:84 , srv1194489305:84 , srv1242045651:84 , srv1349824585:84 , srv1502180515:84 , srv1502462805:84 , srv1537889511:84 , srv1551478433:84 , srv1577844259:84 , srv1601958875:84 , srv1621519136:84 , srv1638874755:84 , srv1653414266:84 , srv1654183980:84 , srv1777969007:84 , srv1786928863:84 , srv1874390986:84 , srv1955649062:84 , srv1989849870:84 , srv2035105130:84 , srv2046508418:84 , srv2074221824:84 , srv2078328648:84 , srv251823120:84 , srv372521837:84 , srv403463459:84 , srv424847848:84 , srv433518298:84 , srv434622083:84 , srv446068126:84 , srv535801785:84 , srv54286418:84 , srv624063149:84 , srv636153171:84 , srv663368223:84 , srv690159073:84 , srv701561508:84 , srv703023535:84 , srv804780671:84 , srv818082709:84 , srv892347169:84 , srv988921495:84 , srv995151903:84 , srv1081706253:85 , srv1102971773:85 , srv1111795145:85 , srv1136466995:85 , srv1161686194:85 , srv123891052:85 , srv1252785698:85 , srv1320695810:85 , srv1340458023:85 , srv1371921100:85 , srv1416102381:85 , srv141905368:85 , srv1421441163:85 , srv143782906:85 , srv1521010423:85 , srv1583996552:85 , srv1594557012:85 , srv16536197:85 , srv1674085559:85 , srv172751691:85 , srv1764086977:85 , srv1808020694:85 , srv1879549038:85 , srv1896572343:85 , srv1949395799:85 , srv1966307453:85 , srv1982178250:85 , srv2031776324:85 , srv2044207070:85 , srv215175938:85 , srv244703426:85 , srv303277950:85 , srv353981903:85 , srv368460176:85 , srv419377234:85 , srv515493474:85 , srv546874368:85 , srv598977505:85 , srv602337233:85 , srv618720744:85 , srv675392462:85 , srv677217386:85 , srv677338002:85 , srv715291162:85 , srv774021411:85 , srv886080826:85 , srv920084929:85 , srv1753669075:86 , srv2106710287:86 , srv7749794:86 , srv1961707590:633 }
2024-02-04 06:53:12,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-04 06:53:12,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1416102381=20, srv1194489305=12, srv353981903=65, srv303277950=64, srv1036499633=1, srv1018024614=0, srv818082709=94, srv515493474=74, srv123891052=13, srv703023535=89, srv1638874755=34, srv1786928863=43, srv141905368=21, srv1349824585=18, srv1076041235=3, srv1577844259=29, srv215175938=61, srv434622083=72, srv1077680278=4, srv1242045651=14, srv251823120=63, srv1949395799=48, srv618720744=80, srv368460176=66, srv988921495=98, srv1654183980=37, srv701561508=88, srv1537889511=27, srv1621519136=33, srv2078328648=59, srv675392462=84, srv172751691=39, srv372521837=67, srv715291162=90, srv1102971773=6, srv1874390986=45, srv1594557012=31, srv2106710287=60, srv1777969007=42, srv1989849870=53, srv403463459=68, srv1081706253=5, srv602337233=79, srv1896572343=47, srv1521010423=26, srv1252785698=15, srv677217386=85, srv143782906=23, srv1551478433=28, srv1961707590=50, srv2035105130=55, srv920084929=97, srv2031776324=54, srv2044207070=56, srv546874368=77, srv446068126=73, srv690159073=87, srv804780671=93, srv1421441163=22, srv995151903=99, srv2074221824=58, srv1753669075=40, srv424847848=70, srv244703426=62, srv886080826=95, srv1982178250=52, srv1502462805=25, srv1340458023=17, srv1966307453=51, srv1808020694=44, srv7749794=92, srv1111795145=7, srv2046508418=57, srv1955649062=49, srv1161686194=10, srv1320695810=16, srv1371921100=19, srv419377234=69, srv774021411=91, srv1653414266=35, srv663368223=83, srv1674085559=38, srv1136466995=8, srv54286418=76, srv535801785=75, srv1601958875=32, srv636153171=82, srv624063149=81, srv677338002=86, srv16536197=36, srv1583996552=30, srv433518298=71, srv1879549038=46, srv598977505=78, srv1043894469=2, srv1156630340=9, srv892347169=96, srv1502180515=24, srv1764086977=41, srv1170333804=11} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-04 06:53:12,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-04 06:53:12,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-04 06:53:12,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-04 06:53:12,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-04 06:53:12,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-04 06:53:12,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-04 06:53:12,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-04 06:53:12,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-04 06:53:12,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-04 06:53:12,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-04 06:53:12,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-04 06:53:12,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-04 06:53:12,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-04 06:53:12,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-04 06:53:12,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-04 06:53:12,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-04 06:53:12,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-04 06:53:12,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-04 06:53:12,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-04 06:53:12,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-04 06:53:12,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-04 06:53:12,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-04 06:53:12,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-04 06:53:12,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-04 06:53:12,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-04 06:53:12,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-04 06:53:12,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-04 06:53:12,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-04 06:53:12,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-04 06:53:12,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-04 06:53:12,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-02-04 06:53:12,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-02-04 06:53:12,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-02-04 06:53:12,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-02-04 06:53:12,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-02-04 06:53:12,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-02-04 06:53:12,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-02-04 06:53:12,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-02-04 06:53:12,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-02-04 06:53:12,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-02-04 06:53:12,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-02-04 06:53:12,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-02-04 06:53:12,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-02-04 06:53:12,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-02-04 06:53:12,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-02-04 06:53:12,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-02-04 06:53:12,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-02-04 06:53:12,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-02-04 06:53:12,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-02-04 06:53:12,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-02-04 06:53:12,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-02-04 06:53:12,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-02-04 06:53:12,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-02-04 06:53:12,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-02-04 06:53:12,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-02-04 06:53:12,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-02-04 06:53:12,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-02-04 06:53:12,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-02-04 06:53:12,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-02-04 06:53:12,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-02-04 06:53:12,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-02-04 06:53:12,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-02-04 06:53:12,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-02-04 06:53:12,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-02-04 06:53:12,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-02-04 06:53:12,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-02-04 06:53:12,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-02-04 06:53:12,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-02-04 06:53:12,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-02-04 06:53:12,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-02-04 06:53:12,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-02-04 06:53:12,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-02-04 06:53:12,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-02-04 06:53:12,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-02-04 06:53:12,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-02-04 06:53:12,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-02-04 06:53:12,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-02-04 06:53:12,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-02-04 06:53:12,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-02-04 06:53:12,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-02-04 06:53:12,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-02-04 06:53:12,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-02-04 06:53:12,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-02-04 06:53:12,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-02-04 06:53:12,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-02-04 06:53:12,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-02-04 06:53:12,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-02-04 06:53:12,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-02-04 06:53:12,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-02-04 06:53:12,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-02-04 06:53:12,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-02-04 06:53:12,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-02-04 06:53:12,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-02-04 06:53:12,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-02-04 06:53:12,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-02-04 06:53:12,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-02-04 06:53:12,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-02-04 06:53:12,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-02-04 06:53:12,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-02-04 06:53:12,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-02-04 06:53:12,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-02-04 06:53:12,369 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=2.023999072597102E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.868392354571063E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0024158102511315956); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.868392354571063E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-04 06:53:12,370 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1043894469:83 , srv1156630340:83 , srv1018024614:84 , srv1036499633:84 , srv1076041235:84 , srv1077680278:84 , srv1170333804:84 , srv1194489305:84 , srv1242045651:84 , srv1349824585:84 , srv1502180515:84 , srv1502462805:84 , srv1537889511:84 , srv1551478433:84 , srv1577844259:84 , srv1601958875:84 , srv1621519136:84 , srv1638874755:84 , srv1653414266:84 , srv1654183980:84 , srv1777969007:84 , srv1786928863:84 , srv1874390986:84 , srv1955649062:84 , srv1989849870:84 , srv2035105130:84 , srv2046508418:84 , srv2074221824:84 , srv2078328648:84 , srv251823120:84 , srv372521837:84 , srv403463459:84 , srv424847848:84 , srv433518298:84 , srv434622083:84 , srv446068126:84 , srv535801785:84 , srv54286418:84 , srv624063149:84 , srv636153171:84 , srv663368223:84 , srv690159073:84 , srv701561508:84 , srv703023535:84 , srv804780671:84 , srv818082709:84 , srv892347169:84 , srv988921495:84 , srv995151903:84 , srv1081706253:85 , srv1102971773:85 , srv1111795145:85 , srv1136466995:85 , srv1161686194:85 , srv123891052:85 , srv1252785698:85 , srv1320695810:85 , srv1340458023:85 , srv1371921100:85 , srv1416102381:85 , srv141905368:85 , srv1421441163:85 , srv143782906:85 , srv1521010423:85 , srv1583996552:85 , srv1594557012:85 , srv16536197:85 , srv1674085559:85 , srv172751691:85 , srv1764086977:85 , srv1808020694:85 , srv1879549038:85 , srv1896572343:85 , srv1949395799:85 , srv1966307453:85 , srv1982178250:85 , srv2031776324:85 , srv2044207070:85 , srv215175938:85 , srv244703426:85 , srv303277950:85 , srv353981903:85 , srv368460176:85 , srv419377234:85 , srv515493474:85 , srv546874368:85 , srv598977505:85 , srv602337233:85 , srv618720744:85 , srv675392462:85 , srv677217386:85 , srv677338002:85 , srv715291162:85 , srv774021411:85 , srv886080826:85 , srv920084929:85 , srv1753669075:86 , srv2106710287:86 , srv7749794:86 , srv1961707590:633 }
2024-02-04 06:53:12,424 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=273 (was 273), ProcessCount=169 (was 169), AvailableMemoryMB=2368 (was 3173)
2024-02-04 06:53:12,439 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=273, ProcessCount=169, AvailableMemoryMB=2367
2024-02-04 06:53:12,440 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-02-04 06:53:12,440 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2024-02-04 06:53:12,452 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1034337691:84 , srv1057096881:84 , srv1191465843:84 , srv125849214:84 , srv1352545892:84 , srv1375372638:84 , srv1380498993:84 , srv1408368155:84 , srv1409435380:84 , srv1438975647:84 , srv1635992435:84 , srv1670204691:84 , srv1825004812:84 , srv184971730:84 , srv1965847229:84 , srv2035437655:84 , srv2096369260:84 , srv2097118574:84 , srv260052154:84 , srv338577626:84 , srv36481245:84 , srv422347495:84 , srv459199183:84 , srv519545823:84 , srv606854296:84 , srv71236977:84 , srv770747745:84 , srv808584893:84 , srv961603662:84 , srv315236705:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2024-02-04 06:53:12,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-04 06:53:12,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv808584893=28, srv1057096881=1, srv1375372638=5, srv1380498993=6, srv260052154=18, srv1409435380=8, srv36481245=21, srv125849214=3, srv2096369260=16, srv1408368155=7, srv315236705=19, srv961603662=29, srv1034337691=0, srv71236977=26, srv1352545892=4, srv459199183=23, srv1191465843=2, srv2097118574=17, srv1965847229=14, srv422347495=22, srv519545823=24, srv1635992435=10, srv2035437655=15, srv338577626=20, srv1438975647=9, srv1825004812=12, srv1670204691=11, srv184971730=13, srv606854296=25, srv770747745=27} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-04 06:53:12,465 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-04 06:53:12,465 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-04 06:53:12,465 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-04 06:53:12,465 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-04 06:53:12,465 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-04 06:53:12,465 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-04 06:53:12,465 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-04 06:53:12,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-04 06:53:12,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-04 06:53:12,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-04 06:53:12,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-04 06:53:12,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-04 06:53:12,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-04 06:53:12,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-04 06:53:12,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-04 06:53:12,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-04 06:53:12,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-04 06:53:12,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-04 06:53:12,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-04 06:53:12,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-04 06:53:12,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-04 06:53:12,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-04 06:53:12,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-04 06:53:12,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-04 06:53:12,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-04 06:53:12,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-04 06:53:12,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-04 06:53:12,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-04 06:53:12,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-04 06:53:12,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-04 06:53:12,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-02-04 06:53:12,490 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1748924627978838, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-04 06:53:12,490 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.991274412370379, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1748924627978838, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2024-02-04 06:55:12,472 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 7150816 different iterations.  Found a solution that moves 1858 regions; Going from a computed imbalance of 0.991274412370379 to a new imbalance of 4.336664689392977E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6881481481481482, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-04 06:55:12,478 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1034337691:90 , srv1057096881:90 , srv1191465843:90 , srv125849214:90 , srv1352545892:90 , srv1375372638:90 , srv1380498993:90 , srv1408368155:90 , srv1409435380:90 , srv1438975647:90 , srv1635992435:90 , srv1670204691:90 , srv1825004812:90 , srv184971730:90 , srv1965847229:90 , srv2035437655:90 , srv2096369260:90 , srv2097118574:90 , srv260052154:90 , srv315236705:90 , srv338577626:90 , srv36481245:90 , srv422347495:90 , srv459199183:90 , srv519545823:90 , srv606854296:90 , srv71236977:90 , srv770747745:90 , srv808584893:90 , srv961603662:90 }
2024-02-04 06:55:12,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-04 06:55:12,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv808584893=28, srv1057096881=1, srv1375372638=5, srv1380498993=6, srv260052154=18, srv1409435380=8, srv36481245=21, srv125849214=3, srv2096369260=16, srv1408368155=7, srv315236705=19, srv961603662=29, srv1034337691=0, srv71236977=26, srv1352545892=4, srv459199183=23, srv1191465843=2, srv2097118574=17, srv1965847229=14, srv422347495=22, srv519545823=24, srv1635992435=10, srv2035437655=15, srv338577626=20, srv1438975647=9, srv1825004812=12, srv1670204691=11, srv184971730=13, srv606854296=25, srv770747745=27} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-04 06:55:12,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-04 06:55:12,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-04 06:55:12,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-04 06:55:12,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-04 06:55:12,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-04 06:55:12,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-04 06:55:12,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-04 06:55:12,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-04 06:55:12,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-04 06:55:12,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-04 06:55:12,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-04 06:55:12,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-04 06:55:12,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-04 06:55:12,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-04 06:55:12,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-04 06:55:12,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-04 06:55:12,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-04 06:55:12,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-04 06:55:12,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-04 06:55:12,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-04 06:55:12,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-04 06:55:12,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-04 06:55:12,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-04 06:55:12,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-04 06:55:12,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-04 06:55:12,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-04 06:55:12,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-04 06:55:12,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-04 06:55:12,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-04 06:55:12,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-04 06:55:12,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-02-04 06:55:12,502 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-04 06:55:12,502 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1034337691:90 , srv1057096881:90 , srv1191465843:90 , srv125849214:90 , srv1352545892:90 , srv1375372638:90 , srv1380498993:90 , srv1408368155:90 , srv1409435380:90 , srv1438975647:90 , srv1635992435:90 , srv1670204691:90 , srv1825004812:90 , srv184971730:90 , srv1965847229:90 , srv2035437655:90 , srv2096369260:90 , srv2097118574:90 , srv260052154:90 , srv315236705:90 , srv338577626:90 , srv36481245:90 , srv422347495:90 , srv459199183:90 , srv519545823:90 , srv606854296:90 , srv71236977:90 , srv770747745:90 , srv808584893:90 , srv961603662:90 }
2024-02-04 06:55:12,517 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=183 (was 273), ProcessCount=169 (was 169), AvailableMemoryMB=2036 (was 2367)