2023-10-14 00:53:29,177 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-10-14 00:53:29,519 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-10-14 00:53:29,632 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-10-14 00:53:29,701 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-10-14 00:53:29,711 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-10-14 00:53:29,726 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=340, ProcessCount=171, AvailableMemoryMB=2570
2023-10-14 00:53:29,728 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-10-14 00:53:29,729 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-10-14 00:53:29,889 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1000243526:84 , srv100090997:84 , srv1002928390:84 , srv1043000076:84 , srv1044739968:84 , srv1093328710:84 , srv1108618014:84 , srv1109175963:84 , srv1111026117:84 , srv1133580228:84 , srv1173469036:84 , srv1188749359:84 , srv1204296732:84 , srv1205188417:84 , srv1207851542:84 , srv1211212468:84 , srv123205132:84 , srv1247054110:84 , srv1265427343:84 , srv1337272893:84 , srv1346812217:84 , srv1367642307:84 , srv139086554:84 , srv1403506442:84 , srv1407565327:84 , srv1458404313:84 , srv1459720030:84 , srv1473716158:84 , srv1493351432:84 , srv156917038:84 , srv1572382530:84 , srv1614961203:84 , srv1635291746:84 , srv1663342854:84 , srv1663811545:84 , srv1715831675:84 , srv1716697217:84 , srv1744281039:84 , srv1749860048:84 , srv1765484100:84 , srv1820045757:84 , srv1864489517:84 , srv1884307367:84 , srv1921311973:84 , srv1925512530:84 , srv1937100842:84 , srv1948187758:84 , srv1953335801:84 , srv1973509293:84 , srv1979412821:84 , srv1981310527:84 , srv1995669911:84 , srv1998455494:84 , srv2008901169:84 , srv2019752345:84 , srv2046484911:84 , srv2049696164:84 , srv2075531847:84 , srv2107534995:84 , srv2124678756:84 , srv225384983:84 , srv230779502:84 , srv258541857:84 , srv263028472:84 , srv27990575:84 , srv30067588:84 , srv303483111:84 , srv319339784:84 , srv329119027:84 , srv329221003:84 , srv33216590:84 , srv355327683:84 , srv370161891:84 , srv381493749:84 , srv382340911:84 , srv41499423:84 , srv425738086:84 , srv429111503:84 , srv448094178:84 , srv481956503:84 , srv504323470:84 , srv531160177:84 , srv585139685:84 , srv590446571:84 , srv616428661:84 , srv632624203:84 , srv659326334:84 , srv690494379:84 , srv725496825:84 , srv744982598:84 , srv781544391:84 , srv78671100:84 , srv846892939:84 , srv852537470:84 , srv85910997:84 , srv905281274:84 , srv949956738:84 , srv976876030:84 , srv998574084:84 , srv494985933:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-10-14 00:53:29,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-14 00:53:29,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1367642307=21, srv370161891=72, srv1093328710=5, srv30067588=65, srv531160177=82, srv1188749359=11, srv263028472=63, srv1979412821=49, srv616428661=85, srv329119027=68, srv1205188417=13, srv1044739968=4, srv846892939=93, srv494985933=80, srv381493749=73, srv1407565327=24, srv1207851542=14, srv659326334=87, srv2019752345=54, srv382340911=74, srv632624203=86, srv425738086=76, srv1265427343=18, srv303483111=66, srv1133580228=9, srv1921311973=43, srv585139685=83, srv329221003=69, srv1000243526=0, srv319339784=67, srv1973509293=48, srv1925512530=44, srv123205132=16, srv230779502=61, srv1002928390=2, srv1204296732=12, srv1459720030=26, srv429111503=77, srv1749860048=38, srv139086554=22, srv1403506442=23, srv852537470=94, srv481956503=79, srv1716697217=36, srv905281274=96, srv1493351432=28, srv504323470=81, srv725496825=89, srv2124678756=59, srv1346812217=20, srv78671100=92, srv1820045757=40, srv976876030=98, srv1247054110=17, srv41499423=75, srv100090997=1, srv1473716158=27, srv258541857=62, srv1572382530=30, srv1948187758=46, srv781544391=91, srv1108618014=6, srv2046484911=55, srv1663342854=33, srv690494379=88, srv1765484100=39, srv2075531847=57, srv1211212468=15, srv1937100842=45, srv1337272893=19, srv744982598=90, srv156917038=29, srv85910997=95, srv2049696164=56, srv1744281039=37, srv1995669911=51, srv1981310527=50, srv998574084=99, srv1458404313=25, srv1614961203=31, srv1998455494=52, srv2008901169=53, srv27990575=64, srv1953335801=47, srv1635291746=32, srv1884307367=42, srv1663811545=34, srv1715831675=35, srv590446571=84, srv1109175963=7, srv2107534995=58, srv448094178=78, srv1111026117=8, srv949956738=97, srv225384983=60, srv1043000076=3, srv1173469036=10, srv1864489517=41, srv355327683=71, srv33216590=70} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-14 00:53:30,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-14 00:53:30,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-14 00:53:30,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-14 00:53:30,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-14 00:53:30,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-14 00:53:30,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-14 00:53:30,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-14 00:53:30,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-14 00:53:30,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-14 00:53:30,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-14 00:53:30,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-14 00:53:30,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-14 00:53:30,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-14 00:53:30,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-14 00:53:30,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-14 00:53:30,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-14 00:53:30,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-14 00:53:30,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-14 00:53:30,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-14 00:53:30,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-14 00:53:30,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-14 00:53:30,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-14 00:53:30,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-14 00:53:30,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-14 00:53:30,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-14 00:53:30,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-14 00:53:30,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-14 00:53:30,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-14 00:53:30,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-14 00:53:30,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-14 00:53:30,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-14 00:53:30,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-14 00:53:30,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-14 00:53:30,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-14 00:53:30,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-14 00:53:30,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-14 00:53:30,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-14 00:53:30,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-14 00:53:30,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-14 00:53:30,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-14 00:53:30,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-14 00:53:30,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-14 00:53:30,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-14 00:53:30,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-14 00:53:30,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-14 00:53:30,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-14 00:53:30,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-14 00:53:30,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-14 00:53:30,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-14 00:53:30,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-14 00:53:30,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-14 00:53:30,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-14 00:53:30,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-14 00:53:30,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-14 00:53:30,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-14 00:53:30,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-14 00:53:30,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-14 00:53:30,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-14 00:53:30,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-14 00:53:30,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-14 00:53:30,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-14 00:53:30,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-14 00:53:30,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-14 00:53:30,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-14 00:53:30,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-14 00:53:30,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-14 00:53:30,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-14 00:53:30,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-14 00:53:30,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-14 00:53:30,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-14 00:53:30,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-14 00:53:30,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-14 00:53:30,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-14 00:53:30,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-14 00:53:30,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-14 00:53:30,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-14 00:53:30,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-14 00:53:30,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-14 00:53:30,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-14 00:53:30,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-14 00:53:30,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-14 00:53:30,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-14 00:53:30,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-14 00:53:30,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-14 00:53:30,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-14 00:53:30,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-14 00:53:30,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-14 00:53:30,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-14 00:53:30,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-14 00:53:30,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-14 00:53:30,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-14 00:53:30,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-14 00:53:30,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-14 00:53:30,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-14 00:53:30,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-14 00:53:30,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-14 00:53:30,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-14 00:53:30,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-14 00:53:30,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-14 00:53:30,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-14 00:53:30,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-14 00:53:30,463 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-14 00:53:30,463 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-14 00:53:30,463 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-14 00:53:31,419 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 1327 ms to try 5000 different iterations.  Found a solution that moves 4697 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1780899050414393. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.041930886905928846, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05954560447544117, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5218888888888888, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.041930886905928846, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17316666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24091666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-14 00:53:31,452 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv659326334:82 , srv976876030:82 , srv1109175963:83 , srv1884307367:83 , srv2008901169:83 , srv381493749:83 , srv448094178:83 , srv1173469036:84 , srv1205188417:84 , srv1211212468:84 , srv1337272893:84 , srv139086554:84 , srv1403506442:84 , srv1614961203:84 , srv1765484100:84 , srv1820045757:84 , srv1995669911:84 , srv2019752345:84 , srv319339784:84 , srv329119027:84 , srv382340911:84 , srv949956738:84 , srv998574084:84 , srv1000243526:85 , srv100090997:85 , srv1204296732:85 , srv1207851542:85 , srv1247054110:85 , srv1458404313:85 , srv1744281039:85 , srv1749860048:85 , srv1925512530:85 , srv1973509293:85 , srv1979412821:85 , srv329221003:85 , srv504323470:85 , srv616428661:85 , srv744982598:85 , srv123205132:86 , srv1346812217:86 , srv1493351432:86 , srv1572382530:86 , srv1663342854:86 , srv1663811545:86 , srv1948187758:86 , srv1998455494:86 , srv2046484911:86 , srv2107534995:86 , srv258541857:86 , srv27990575:86 , srv303483111:86 , srv41499423:86 , srv429111503:86 , srv481956503:86 , srv632624203:86 , srv690494379:86 , srv725496825:86 , srv846892939:86 , srv85910997:86 , srv1093328710:87 , srv1133580228:87 , srv1265427343:87 , srv1367642307:87 , srv1473716158:87 , srv156917038:87 , srv1716697217:87 , srv1921311973:87 , srv1953335801:87 , srv1981310527:87 , srv2049696164:87 , srv225384983:87 , srv230779502:87 , srv355327683:87 , srv781544391:87 , srv1044739968:88 , srv1108618014:88 , srv1111026117:88 , srv1407565327:88 , srv1459720030:88 , srv1715831675:88 , srv2075531847:88 , srv370161891:88 , srv531160177:88 , srv590446571:88 , srv78671100:88 , srv1043000076:89 , srv1188749359:89 , srv1635291746:89 , srv2124678756:89 , srv263028472:89 , srv30067588:89 , srv585139685:89 , srv1002928390:90 , srv33216590:90 , srv852537470:90 , srv1864489517:91 , srv905281274:91 , srv425738086:92 , srv1937100842:93 , srv494985933:463 }
2023-10-14 00:53:31,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-14 00:53:31,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1367642307=21, srv370161891=72, srv1093328710=5, srv30067588=65, srv531160177=82, srv1188749359=11, srv263028472=63, srv1979412821=49, srv616428661=85, srv329119027=68, srv1205188417=13, srv1044739968=4, srv846892939=93, srv494985933=80, srv381493749=73, srv1407565327=24, srv1207851542=14, srv659326334=87, srv2019752345=54, srv382340911=74, srv632624203=86, srv425738086=76, srv1265427343=18, srv303483111=66, srv1133580228=9, srv1921311973=43, srv585139685=83, srv329221003=69, srv1000243526=0, srv319339784=67, srv1973509293=48, srv1925512530=44, srv123205132=16, srv230779502=61, srv1002928390=2, srv1204296732=12, srv1459720030=26, srv429111503=77, srv1749860048=38, srv139086554=22, srv1403506442=23, srv852537470=94, srv481956503=79, srv1716697217=36, srv905281274=96, srv1493351432=28, srv504323470=81, srv725496825=89, srv2124678756=59, srv1346812217=20, srv78671100=92, srv1820045757=40, srv976876030=98, srv1247054110=17, srv41499423=75, srv100090997=1, srv1473716158=27, srv258541857=62, srv1572382530=30, srv1948187758=46, srv781544391=91, srv1108618014=6, srv2046484911=55, srv1663342854=33, srv690494379=88, srv1765484100=39, srv2075531847=57, srv1211212468=15, srv1937100842=45, srv1337272893=19, srv744982598=90, srv156917038=29, srv85910997=95, srv2049696164=56, srv1744281039=37, srv1995669911=51, srv1981310527=50, srv998574084=99, srv1458404313=25, srv1614961203=31, srv1998455494=52, srv2008901169=53, srv27990575=64, srv1953335801=47, srv1635291746=32, srv1884307367=42, srv1663811545=34, srv1715831675=35, srv590446571=84, srv1109175963=7, srv2107534995=58, srv448094178=78, srv1111026117=8, srv949956738=97, srv225384983=60, srv1043000076=3, srv1173469036=10, srv1864489517=41, srv355327683=71, srv33216590=70} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-14 00:53:31,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-14 00:53:31,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-14 00:53:31,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-14 00:53:31,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-14 00:53:31,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-14 00:53:31,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-14 00:53:31,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-14 00:53:31,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-14 00:53:31,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-14 00:53:31,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-14 00:53:31,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-14 00:53:31,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-14 00:53:31,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-14 00:53:31,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-14 00:53:31,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-14 00:53:31,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-14 00:53:31,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-14 00:53:31,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-14 00:53:31,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-14 00:53:31,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-14 00:53:31,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-14 00:53:31,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-14 00:53:31,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-14 00:53:31,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-14 00:53:31,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-14 00:53:31,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-14 00:53:31,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-14 00:53:31,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-14 00:53:31,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-14 00:53:31,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-14 00:53:31,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-14 00:53:31,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-14 00:53:31,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-14 00:53:31,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-14 00:53:31,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-14 00:53:31,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-14 00:53:31,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-14 00:53:31,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-14 00:53:31,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-14 00:53:31,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-14 00:53:31,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-14 00:53:31,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-14 00:53:31,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-14 00:53:31,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-14 00:53:31,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-14 00:53:31,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-14 00:53:31,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-14 00:53:31,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-14 00:53:31,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-14 00:53:31,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-14 00:53:31,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-14 00:53:31,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-14 00:53:31,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-14 00:53:31,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-14 00:53:31,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-14 00:53:31,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-14 00:53:31,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-14 00:53:31,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-14 00:53:31,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-14 00:53:31,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-14 00:53:31,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-14 00:53:31,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-14 00:53:31,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-14 00:53:31,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-14 00:53:31,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-14 00:53:31,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-14 00:53:31,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-14 00:53:31,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-14 00:53:31,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-14 00:53:31,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-14 00:53:31,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-14 00:53:31,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-14 00:53:31,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-14 00:53:31,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-14 00:53:31,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-14 00:53:31,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-14 00:53:31,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-14 00:53:31,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-14 00:53:31,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-14 00:53:31,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-14 00:53:31,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-14 00:53:31,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-14 00:53:31,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-14 00:53:31,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-14 00:53:31,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-14 00:53:31,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-14 00:53:31,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-14 00:53:31,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-14 00:53:31,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-14 00:53:31,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-14 00:53:31,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-14 00:53:31,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-14 00:53:31,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-14 00:53:31,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-14 00:53:31,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-14 00:53:31,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-14 00:53:31,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-14 00:53:31,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-14 00:53:31,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-14 00:53:31,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-14 00:53:31,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-14 00:53:31,694 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.041930886905928846, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05954560447544117, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.041930886905928846, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17316666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24091666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-14 00:53:31,694 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-14 00:53:31,695 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.1780570159444865, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.041930886905928846, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05954560447544117, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.041930886905928846, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17316666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24091666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-14 00:53:32,010 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 494 ms to try 5000 different iterations.  Found a solution that moves 2591 regions; Going from a computed imbalance of 0.1780570159444865 to a new imbalance of 0.014656268482607376. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022497176666746645); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.031179578340629285, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2878888888888889, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022497176666746645); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01175); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.042333333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-14 00:53:32,024 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1937100842:78 , srv1864489517:79 , srv425738086:81 , srv852537470:81 , srv1002928390:82 , srv1044739968:82 , srv370161891:82 , srv590446571:82 , srv905281274:82 , srv1043000076:83 , srv1407565327:83 , srv1459720030:83 , srv263028472:83 , srv30067588:83 , srv1093328710:84 , srv1111026117:84 , srv1188749359:84 , srv156917038:84 , srv1635291746:84 , srv1663342854:84 , srv1716697217:84 , srv2046484911:84 , srv2049696164:84 , srv2124678756:84 , srv225384983:84 , srv27990575:84 , srv481956503:84 , srv725496825:84 , srv100090997:85 , srv1133580228:85 , srv1204296732:85 , srv1265427343:85 , srv1346812217:85 , srv1473716158:85 , srv1572382530:85 , srv1663811545:85 , srv1715831675:85 , srv1749860048:85 , srv1981310527:85 , srv2075531847:85 , srv258541857:85 , srv33216590:85 , srv429111503:85 , srv632624203:85 , srv781544391:85 , srv846892939:85 , srv1000243526:86 , srv1108618014:86 , srv1205188417:86 , srv1211212468:86 , srv123205132:86 , srv1744281039:86 , srv1948187758:86 , srv1973509293:86 , srv1979412821:86 , srv1998455494:86 , srv2107534995:86 , srv303483111:86 , srv355327683:86 , srv41499423:86 , srv531160177:86 , srv78671100:86 , srv949956738:86 , srv1207851542:87 , srv1247054110:87 , srv1367642307:87 , srv1493351432:87 , srv1820045757:87 , srv1921311973:87 , srv1953335801:87 , srv230779502:87 , srv329119027:87 , srv329221003:87 , srv585139685:87 , srv690494379:87 , srv1109175963:88 , srv1173469036:88 , srv139086554:88 , srv1403506442:88 , srv1458404313:88 , srv1884307367:88 , srv1925512530:88 , srv319339784:88 , srv504323470:88 , srv616428661:88 , srv744982598:88 , srv998574084:88 , srv1614961203:89 , srv1765484100:89 , srv2019752345:89 , srv382340911:89 , srv448094178:89 , srv85910997:89 , srv1337272893:90 , srv1995669911:90 , srv381493749:90 , srv659326334:90 , srv2008901169:91 , srv976876030:92 , srv494985933:511 }
2023-10-14 00:53:32,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-14 00:53:32,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1367642307=21, srv370161891=72, srv1093328710=5, srv30067588=65, srv531160177=82, srv1188749359=11, srv263028472=63, srv1979412821=49, srv616428661=85, srv329119027=68, srv1205188417=13, srv1044739968=4, srv846892939=93, srv494985933=80, srv381493749=73, srv1407565327=24, srv1207851542=14, srv659326334=87, srv2019752345=54, srv382340911=74, srv632624203=86, srv425738086=76, srv1265427343=18, srv303483111=66, srv1133580228=9, srv1921311973=43, srv585139685=83, srv329221003=69, srv1000243526=0, srv319339784=67, srv1973509293=48, srv1925512530=44, srv123205132=16, srv230779502=61, srv1002928390=2, srv1204296732=12, srv1459720030=26, srv429111503=77, srv1749860048=38, srv139086554=22, srv1403506442=23, srv852537470=94, srv481956503=79, srv1716697217=36, srv905281274=96, srv1493351432=28, srv504323470=81, srv725496825=89, srv2124678756=59, srv1346812217=20, srv78671100=92, srv1820045757=40, srv976876030=98, srv1247054110=17, srv41499423=75, srv100090997=1, srv1473716158=27, srv258541857=62, srv1572382530=30, srv1948187758=46, srv781544391=91, srv1108618014=6, srv2046484911=55, srv1663342854=33, srv690494379=88, srv1765484100=39, srv2075531847=57, srv1211212468=15, srv1937100842=45, srv1337272893=19, srv744982598=90, srv156917038=29, srv85910997=95, srv2049696164=56, srv1744281039=37, srv1995669911=51, srv1981310527=50, srv998574084=99, srv1458404313=25, srv1614961203=31, srv1998455494=52, srv2008901169=53, srv27990575=64, srv1953335801=47, srv1635291746=32, srv1884307367=42, srv1663811545=34, srv1715831675=35, srv590446571=84, srv1109175963=7, srv2107534995=58, srv448094178=78, srv1111026117=8, srv949956738=97, srv225384983=60, srv1043000076=3, srv1173469036=10, srv1864489517=41, srv355327683=71, srv33216590=70} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-14 00:53:32,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-14 00:53:32,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-14 00:53:32,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-14 00:53:32,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-14 00:53:32,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-14 00:53:32,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-14 00:53:32,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-14 00:53:32,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-14 00:53:32,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-14 00:53:32,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-14 00:53:32,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-14 00:53:32,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-14 00:53:32,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-14 00:53:32,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-14 00:53:32,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-14 00:53:32,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-14 00:53:32,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-14 00:53:32,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-14 00:53:32,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-14 00:53:32,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-14 00:53:32,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-14 00:53:32,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-14 00:53:32,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-14 00:53:32,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-14 00:53:32,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-14 00:53:32,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-14 00:53:32,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-14 00:53:32,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-14 00:53:32,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-14 00:53:32,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-14 00:53:32,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-14 00:53:32,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-14 00:53:32,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-14 00:53:32,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-14 00:53:32,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-14 00:53:32,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-14 00:53:32,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-14 00:53:32,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-14 00:53:32,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-14 00:53:32,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-14 00:53:32,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-14 00:53:32,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-14 00:53:32,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-14 00:53:32,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-14 00:53:32,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-14 00:53:32,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-14 00:53:32,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-14 00:53:32,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-14 00:53:32,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-14 00:53:32,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-14 00:53:32,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-14 00:53:32,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-14 00:53:32,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-14 00:53:32,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-14 00:53:32,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-14 00:53:32,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-14 00:53:32,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-14 00:53:32,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-14 00:53:32,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-14 00:53:32,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-14 00:53:32,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-14 00:53:32,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-14 00:53:32,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-14 00:53:32,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-14 00:53:32,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-14 00:53:32,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-14 00:53:32,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-14 00:53:32,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-14 00:53:32,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-14 00:53:32,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-14 00:53:32,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-14 00:53:32,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-14 00:53:32,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-14 00:53:32,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-14 00:53:32,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-14 00:53:32,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-14 00:53:32,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-14 00:53:32,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-14 00:53:32,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-14 00:53:32,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-14 00:53:32,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-14 00:53:32,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-14 00:53:32,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-14 00:53:32,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-14 00:53:32,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-14 00:53:32,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-14 00:53:32,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-14 00:53:32,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-14 00:53:32,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-14 00:53:32,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-14 00:53:32,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-14 00:53:32,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-14 00:53:32,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-14 00:53:32,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-14 00:53:32,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-14 00:53:32,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-14 00:53:32,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-14 00:53:32,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-14 00:53:32,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-14 00:53:32,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-14 00:53:32,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-14 00:53:32,198 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022497176666746645); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.031179578340629285, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022497176666746645); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01175); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.042333333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-14 00:53:32,199 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-14 00:53:32,199 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.014638125912838458, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022497176666746645); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.031179578340629285, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022497176666746645); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01175); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.042333333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-14 00:53:32,417 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 333 ms to try 5000 different iterations.  Found a solution that moves 977 regions; Going from a computed imbalance of 0.014638125912838458 to a new imbalance of 4.423165090323625E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006536324013726713); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00808207060844344); MoveCostFunction : (multiplier=7.0, imbalance=0.10855555555555556, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006536324013726713); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00325); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-14 00:53:32,419 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv585139685:82 , srv1108618014:83 , srv1337272893:83 , srv1921311973:83 , srv1995669911:83 , srv33216590:83 , srv531160177:83 , srv85910997:83 , srv1367642307:84 , srv1458404313:84 , srv1614961203:84 , srv1635291746:84 , srv1715831675:84 , srv1765484100:84 , srv1953335801:84 , srv2008901169:84 , srv2124678756:84 , srv230779502:84 , srv382340911:84 , srv425738086:84 , srv504323470:84 , srv690494379:84 , srv78671100:84 , srv976876030:84 , srv1043000076:85 , srv1111026117:85 , srv1133580228:85 , srv1173469036:85 , srv1188749359:85 , srv123205132:85 , srv1247054110:85 , srv139086554:85 , srv1403506442:85 , srv1493351432:85 , srv1925512530:85 , srv1998455494:85 , srv2019752345:85 , srv2075531847:85 , srv2107534995:85 , srv263028472:85 , srv30067588:85 , srv303483111:85 , srv319339784:85 , srv355327683:85 , srv381493749:85 , srv41499423:85 , srv616428661:85 , srv659326334:85 , srv744982598:85 , srv905281274:85 , srv998574084:85 , srv1000243526:86 , srv1002928390:86 , srv1093328710:86 , srv1207851542:86 , srv1265427343:86 , srv1346812217:86 , srv1459720030:86 , srv1473716158:86 , srv156917038:86 , srv1716697217:86 , srv1744281039:86 , srv1749860048:86 , srv1820045757:86 , srv1884307367:86 , srv1937100842:86 , srv1948187758:86 , srv1973509293:86 , srv1981310527:86 , srv2049696164:86 , srv225384983:86 , srv329221003:86 , srv448094178:86 , srv781544391:86 , srv846892939:86 , srv852537470:86 , srv100090997:87 , srv1044739968:87 , srv1109175963:87 , srv1204296732:87 , srv1205188417:87 , srv1407565327:87 , srv1572382530:87 , srv1663342854:87 , srv1663811545:87 , srv1979412821:87 , srv258541857:87 , srv27990575:87 , srv329119027:87 , srv370161891:87 , srv429111503:87 , srv481956503:87 , srv590446571:87 , srv632624203:87 , srv725496825:87 , srv1211212468:88 , srv1864489517:88 , srv949956738:88 , srv2046484911:89 , srv494985933:542 }
2023-10-14 00:53:32,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-14 00:53:32,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1367642307=21, srv370161891=72, srv1093328710=5, srv30067588=65, srv531160177=82, srv1188749359=11, srv263028472=63, srv1979412821=49, srv616428661=85, srv329119027=68, srv1205188417=13, srv1044739968=4, srv846892939=93, srv494985933=80, srv381493749=73, srv1407565327=24, srv1207851542=14, srv659326334=87, srv2019752345=54, srv382340911=74, srv632624203=86, srv425738086=76, srv1265427343=18, srv303483111=66, srv1133580228=9, srv1921311973=43, srv585139685=83, srv329221003=69, srv1000243526=0, srv319339784=67, srv1973509293=48, srv1925512530=44, srv123205132=16, srv230779502=61, srv1002928390=2, srv1204296732=12, srv1459720030=26, srv429111503=77, srv1749860048=38, srv139086554=22, srv1403506442=23, srv852537470=94, srv481956503=79, srv1716697217=36, srv905281274=96, srv1493351432=28, srv504323470=81, srv725496825=89, srv2124678756=59, srv1346812217=20, srv78671100=92, srv1820045757=40, srv976876030=98, srv1247054110=17, srv41499423=75, srv100090997=1, srv1473716158=27, srv258541857=62, srv1572382530=30, srv1948187758=46, srv781544391=91, srv1108618014=6, srv2046484911=55, srv1663342854=33, srv690494379=88, srv1765484100=39, srv2075531847=57, srv1211212468=15, srv1937100842=45, srv1337272893=19, srv744982598=90, srv156917038=29, srv85910997=95, srv2049696164=56, srv1744281039=37, srv1995669911=51, srv1981310527=50, srv998574084=99, srv1458404313=25, srv1614961203=31, srv1998455494=52, srv2008901169=53, srv27990575=64, srv1953335801=47, srv1635291746=32, srv1884307367=42, srv1663811545=34, srv1715831675=35, srv590446571=84, srv1109175963=7, srv2107534995=58, srv448094178=78, srv1111026117=8, srv949956738=97, srv225384983=60, srv1043000076=3, srv1173469036=10, srv1864489517=41, srv355327683=71, srv33216590=70} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-14 00:53:32,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-14 00:53:32,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-14 00:53:32,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-14 00:53:32,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-14 00:53:32,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-14 00:53:32,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-14 00:53:32,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-14 00:53:32,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-14 00:53:32,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-14 00:53:32,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-14 00:53:32,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-14 00:53:32,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-14 00:53:32,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-14 00:53:32,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-14 00:53:32,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-14 00:53:32,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-14 00:53:32,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-14 00:53:32,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-14 00:53:32,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-14 00:53:32,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-14 00:53:32,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-14 00:53:32,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-14 00:53:32,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-14 00:53:32,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-14 00:53:32,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-14 00:53:32,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-14 00:53:32,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-14 00:53:32,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-14 00:53:32,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-14 00:53:32,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-14 00:53:32,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-14 00:53:32,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-14 00:53:32,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-14 00:53:32,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-14 00:53:32,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-14 00:53:32,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-14 00:53:32,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-14 00:53:32,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-14 00:53:32,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-14 00:53:32,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-14 00:53:32,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-14 00:53:32,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-14 00:53:32,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-14 00:53:32,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-14 00:53:32,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-14 00:53:32,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-14 00:53:32,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-14 00:53:32,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-14 00:53:32,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-14 00:53:32,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-14 00:53:32,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-14 00:53:32,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-14 00:53:32,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-14 00:53:32,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-14 00:53:32,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-14 00:53:32,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-14 00:53:32,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-14 00:53:32,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-14 00:53:32,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-14 00:53:32,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-14 00:53:32,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-14 00:53:32,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-14 00:53:32,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-14 00:53:32,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-14 00:53:32,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-14 00:53:32,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-14 00:53:32,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-14 00:53:32,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-14 00:53:32,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-14 00:53:32,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-14 00:53:32,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-14 00:53:32,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-14 00:53:32,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-14 00:53:32,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-14 00:53:32,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-14 00:53:32,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-14 00:53:32,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-14 00:53:32,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-14 00:53:32,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-14 00:53:32,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-14 00:53:32,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-14 00:53:32,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-14 00:53:32,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-14 00:53:32,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-14 00:53:32,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-14 00:53:32,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-14 00:53:32,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-14 00:53:32,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-14 00:53:32,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-14 00:53:32,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-14 00:53:32,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-14 00:53:32,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-14 00:53:32,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-14 00:53:32,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-14 00:53:32,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-14 00:53:32,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-14 00:53:32,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-14 00:53:32,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-14 00:53:32,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-14 00:53:32,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-14 00:53:32,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-14 00:53:32,543 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006536324013726713); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00808207060844344); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006536324013726713); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00325); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-14 00:53:32,544 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-14 00:53:32,544 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=4.3547540881459573E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006536324013726713); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00808207060844344); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006536324013726713); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00325); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-14 00:53:32,889 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 440 ms to try 5000 different iterations.  Found a solution that moves 225 regions; Going from a computed imbalance of 4.3547540881459573E-4 to a new imbalance of 1.0340626182423434E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=2.2334173672426932E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017082358106351274); MoveCostFunction : (multiplier=7.0, imbalance=0.025); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=2.2334173672426932E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-14 00:53:32,890 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1265427343:83 , srv1981310527:83 , srv2046484911:83 , srv1002928390:84 , srv1109175963:84 , srv1188749359:84 , srv1207851542:84 , srv1211212468:84 , srv1367642307:84 , srv1407565327:84 , srv1473716158:84 , srv1493351432:84 , srv1572382530:84 , srv1663811545:84 , srv1864489517:84 , srv1925512530:84 , srv1948187758:84 , srv1953335801:84 , srv1979412821:84 , srv2008901169:84 , srv2019752345:84 , srv2075531847:84 , srv230779502:84 , srv258541857:84 , srv329119027:84 , srv329221003:84 , srv33216590:84 , srv355327683:84 , srv429111503:84 , srv448094178:84 , srv585139685:84 , srv616428661:84 , srv632624203:84 , srv744982598:84 , srv781544391:84 , srv78671100:84 , srv85910997:84 , srv905281274:84 , srv949956738:84 , srv976876030:84 , srv1000243526:85 , srv100090997:85 , srv1043000076:85 , srv1044739968:85 , srv1093328710:85 , srv1108618014:85 , srv1111026117:85 , srv1133580228:85 , srv1173469036:85 , srv1204296732:85 , srv1205188417:85 , srv123205132:85 , srv1247054110:85 , srv1337272893:85 , srv1346812217:85 , srv139086554:85 , srv1403506442:85 , srv1458404313:85 , srv1459720030:85 , srv156917038:85 , srv1614961203:85 , srv1635291746:85 , srv1663342854:85 , srv1715831675:85 , srv1716697217:85 , srv1744281039:85 , srv1765484100:85 , srv1820045757:85 , srv1884307367:85 , srv1921311973:85 , srv1937100842:85 , srv1973509293:85 , srv1995669911:85 , srv1998455494:85 , srv2049696164:85 , srv2107534995:85 , srv2124678756:85 , srv225384983:85 , srv263028472:85 , srv27990575:85 , srv30067588:85 , srv303483111:85 , srv319339784:85 , srv370161891:85 , srv381493749:85 , srv382340911:85 , srv425738086:85 , srv481956503:85 , srv504323470:85 , srv531160177:85 , srv590446571:85 , srv659326334:85 , srv690494379:85 , srv725496825:85 , srv846892939:85 , srv852537470:85 , srv998574084:85 , srv1749860048:86 , srv41499423:86 , srv494985933:626 }
2023-10-14 00:53:32,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-14 00:53:32,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1367642307=21, srv370161891=72, srv1093328710=5, srv30067588=65, srv531160177=82, srv1188749359=11, srv263028472=63, srv1979412821=49, srv616428661=85, srv329119027=68, srv1205188417=13, srv1044739968=4, srv846892939=93, srv494985933=80, srv381493749=73, srv1407565327=24, srv1207851542=14, srv659326334=87, srv2019752345=54, srv382340911=74, srv632624203=86, srv425738086=76, srv1265427343=18, srv303483111=66, srv1133580228=9, srv1921311973=43, srv585139685=83, srv329221003=69, srv1000243526=0, srv319339784=67, srv1973509293=48, srv1925512530=44, srv123205132=16, srv230779502=61, srv1002928390=2, srv1204296732=12, srv1459720030=26, srv429111503=77, srv1749860048=38, srv139086554=22, srv1403506442=23, srv852537470=94, srv481956503=79, srv1716697217=36, srv905281274=96, srv1493351432=28, srv504323470=81, srv725496825=89, srv2124678756=59, srv1346812217=20, srv78671100=92, srv1820045757=40, srv976876030=98, srv1247054110=17, srv41499423=75, srv100090997=1, srv1473716158=27, srv258541857=62, srv1572382530=30, srv1948187758=46, srv781544391=91, srv1108618014=6, srv2046484911=55, srv1663342854=33, srv690494379=88, srv1765484100=39, srv2075531847=57, srv1211212468=15, srv1937100842=45, srv1337272893=19, srv744982598=90, srv156917038=29, srv85910997=95, srv2049696164=56, srv1744281039=37, srv1995669911=51, srv1981310527=50, srv998574084=99, srv1458404313=25, srv1614961203=31, srv1998455494=52, srv2008901169=53, srv27990575=64, srv1953335801=47, srv1635291746=32, srv1884307367=42, srv1663811545=34, srv1715831675=35, srv590446571=84, srv1109175963=7, srv2107534995=58, srv448094178=78, srv1111026117=8, srv949956738=97, srv225384983=60, srv1043000076=3, srv1173469036=10, srv1864489517=41, srv355327683=71, srv33216590=70} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-14 00:53:32,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-14 00:53:32,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-14 00:53:32,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-14 00:53:32,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-14 00:53:32,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-14 00:53:32,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-14 00:53:32,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-14 00:53:32,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-14 00:53:32,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-14 00:53:32,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-14 00:53:32,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-14 00:53:32,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-14 00:53:32,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-14 00:53:32,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-14 00:53:32,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-14 00:53:32,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-14 00:53:32,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-14 00:53:32,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-14 00:53:32,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-14 00:53:32,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-14 00:53:32,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-14 00:53:32,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-14 00:53:32,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-14 00:53:32,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-14 00:53:32,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-14 00:53:32,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-14 00:53:32,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-14 00:53:32,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-14 00:53:32,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-14 00:53:32,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-14 00:53:32,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-14 00:53:32,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-14 00:53:32,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-14 00:53:32,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-14 00:53:32,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-14 00:53:32,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-14 00:53:32,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-14 00:53:32,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-14 00:53:32,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-14 00:53:32,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-14 00:53:32,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-14 00:53:32,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-14 00:53:32,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-14 00:53:32,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-14 00:53:32,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-14 00:53:32,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-14 00:53:32,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-14 00:53:32,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-14 00:53:32,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-14 00:53:32,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-14 00:53:32,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-14 00:53:32,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-14 00:53:32,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-14 00:53:32,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-14 00:53:32,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-14 00:53:32,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-14 00:53:32,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-14 00:53:32,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-14 00:53:32,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-14 00:53:32,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-14 00:53:32,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-14 00:53:32,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-14 00:53:32,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-14 00:53:32,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-14 00:53:32,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-14 00:53:32,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-14 00:53:32,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-14 00:53:32,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-14 00:53:32,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-14 00:53:32,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-14 00:53:32,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-14 00:53:32,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-14 00:53:32,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-14 00:53:32,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-14 00:53:32,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-14 00:53:32,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-14 00:53:32,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-14 00:53:32,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-14 00:53:32,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-14 00:53:32,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-14 00:53:32,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-14 00:53:32,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-14 00:53:32,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-14 00:53:32,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-14 00:53:32,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-14 00:53:32,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-14 00:53:32,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-14 00:53:32,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-14 00:53:32,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-14 00:53:32,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-14 00:53:32,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-14 00:53:32,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-14 00:53:32,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-14 00:53:32,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-14 00:53:32,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-14 00:53:32,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-14 00:53:32,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-14 00:53:32,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-14 00:53:32,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-14 00:53:32,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-14 00:53:32,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-14 00:53:33,035 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=8.765142508935674E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=2.2334173672426932E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017082358106351274); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=2.2334173672426932E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-14 00:53:33,036 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1265427343:83 , srv1981310527:83 , srv2046484911:83 , srv1002928390:84 , srv1109175963:84 , srv1188749359:84 , srv1207851542:84 , srv1211212468:84 , srv1367642307:84 , srv1407565327:84 , srv1473716158:84 , srv1493351432:84 , srv1572382530:84 , srv1663811545:84 , srv1864489517:84 , srv1925512530:84 , srv1948187758:84 , srv1953335801:84 , srv1979412821:84 , srv2008901169:84 , srv2019752345:84 , srv2075531847:84 , srv230779502:84 , srv258541857:84 , srv329119027:84 , srv329221003:84 , srv33216590:84 , srv355327683:84 , srv429111503:84 , srv448094178:84 , srv585139685:84 , srv616428661:84 , srv632624203:84 , srv744982598:84 , srv781544391:84 , srv78671100:84 , srv85910997:84 , srv905281274:84 , srv949956738:84 , srv976876030:84 , srv1000243526:85 , srv100090997:85 , srv1043000076:85 , srv1044739968:85 , srv1093328710:85 , srv1108618014:85 , srv1111026117:85 , srv1133580228:85 , srv1173469036:85 , srv1204296732:85 , srv1205188417:85 , srv123205132:85 , srv1247054110:85 , srv1337272893:85 , srv1346812217:85 , srv139086554:85 , srv1403506442:85 , srv1458404313:85 , srv1459720030:85 , srv156917038:85 , srv1614961203:85 , srv1635291746:85 , srv1663342854:85 , srv1715831675:85 , srv1716697217:85 , srv1744281039:85 , srv1765484100:85 , srv1820045757:85 , srv1884307367:85 , srv1921311973:85 , srv1937100842:85 , srv1973509293:85 , srv1995669911:85 , srv1998455494:85 , srv2049696164:85 , srv2107534995:85 , srv2124678756:85 , srv225384983:85 , srv263028472:85 , srv27990575:85 , srv30067588:85 , srv303483111:85 , srv319339784:85 , srv370161891:85 , srv381493749:85 , srv382340911:85 , srv425738086:85 , srv481956503:85 , srv504323470:85 , srv531160177:85 , srv590446571:85 , srv659326334:85 , srv690494379:85 , srv725496825:85 , srv846892939:85 , srv852537470:85 , srv998574084:85 , srv1749860048:86 , srv41499423:86 , srv494985933:626 }
2023-10-14 00:53:33,062 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=340 (was 340), ProcessCount=171 (was 171), AvailableMemoryMB=1683 (was 2570)
2023-10-14 00:53:33,071 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=340, ProcessCount=171, AvailableMemoryMB=1682
2023-10-14 00:53:33,072 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-10-14 00:53:33,072 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-10-14 00:53:33,080 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1032630632:84 , srv1161953225:84 , srv1169219308:84 , srv1202873223:84 , srv1222893939:84 , srv1227768654:84 , srv131520823:84 , srv1470018454:84 , srv1493857472:84 , srv1591576387:84 , srv1599229116:84 , srv1608814558:84 , srv1810250614:84 , srv1952473604:84 , srv2076048964:84 , srv209025133:84 , srv2119268426:84 , srv2346969:84 , srv344465465:84 , srv409206569:84 , srv536889466:84 , srv552594602:84 , srv554049766:84 , srv64121835:84 , srv717825470:84 , srv77175318:84 , srv784344373:84 , srv907682222:84 , srv909046918:84 , srv1041461872:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-10-14 00:53:33,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-14 00:53:33,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1493857472=9, srv1041461872=1, srv2076048964=15, srv536889466=21, srv344465465=19, srv1608814558=12, srv552594602=22, srv909046918=29, srv77175318=26, srv131520823=7, srv717825470=25, srv1202873223=4, srv907682222=28, srv1169219308=3, srv1470018454=8, srv1599229116=11, srv409206569=20, srv1810250614=13, srv64121835=24, srv784344373=27, srv1222893939=5, srv1591576387=10, srv1161953225=2, srv554049766=23, srv1032630632=0, srv1227768654=6, srv2346969=18, srv2119268426=17, srv1952473604=14, srv209025133=16} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-14 00:53:33,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-14 00:53:33,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-14 00:53:33,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-14 00:53:33,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-14 00:53:33,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-14 00:53:33,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-14 00:53:33,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-14 00:53:33,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-14 00:53:33,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-14 00:53:33,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-14 00:53:33,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-14 00:53:33,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-14 00:53:33,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-14 00:53:33,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-14 00:53:33,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-14 00:53:33,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-14 00:53:33,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-14 00:53:33,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-14 00:53:33,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-14 00:53:33,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-14 00:53:33,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-14 00:53:33,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-14 00:53:33,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-14 00:53:33,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-14 00:53:33,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-14 00:53:33,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-14 00:53:33,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-14 00:53:33,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-14 00:53:33,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-14 00:53:33,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-14 00:53:33,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-10-14 00:53:33,108 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1426967994752983, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-14 00:53:33,109 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912642676219946, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1426967994752983, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-10-14 00:55:33,096 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6364221 different iterations.  Found a solution that moves 1864 regions; Going from a computed imbalance of 0.9912642676219946 to a new imbalance of 4.3506689887128686E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6903703703703704, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-14 00:55:33,105 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1032630632:90 , srv1041461872:90 , srv1161953225:90 , srv1169219308:90 , srv1202873223:90 , srv1222893939:90 , srv1227768654:90 , srv131520823:90 , srv1470018454:90 , srv1493857472:90 , srv1591576387:90 , srv1599229116:90 , srv1608814558:90 , srv1810250614:90 , srv1952473604:90 , srv2076048964:90 , srv209025133:90 , srv2119268426:90 , srv2346969:90 , srv344465465:90 , srv409206569:90 , srv536889466:90 , srv552594602:90 , srv554049766:90 , srv64121835:90 , srv717825470:90 , srv77175318:90 , srv784344373:90 , srv907682222:90 , srv909046918:90 }
2023-10-14 00:55:33,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-14 00:55:33,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1493857472=9, srv1041461872=1, srv2076048964=15, srv536889466=21, srv344465465=19, srv1608814558=12, srv552594602=22, srv909046918=29, srv77175318=26, srv131520823=7, srv717825470=25, srv1202873223=4, srv907682222=28, srv1169219308=3, srv1470018454=8, srv1599229116=11, srv409206569=20, srv1810250614=13, srv64121835=24, srv784344373=27, srv1222893939=5, srv1591576387=10, srv1161953225=2, srv554049766=23, srv1032630632=0, srv1227768654=6, srv2346969=18, srv2119268426=17, srv1952473604=14, srv209025133=16} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-14 00:55:33,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-14 00:55:33,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-14 00:55:33,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-14 00:55:33,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-14 00:55:33,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-14 00:55:33,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-14 00:55:33,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-14 00:55:33,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-14 00:55:33,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-14 00:55:33,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-14 00:55:33,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-14 00:55:33,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-14 00:55:33,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-14 00:55:33,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-14 00:55:33,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-14 00:55:33,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-14 00:55:33,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-14 00:55:33,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-14 00:55:33,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-14 00:55:33,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-14 00:55:33,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-14 00:55:33,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-14 00:55:33,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-14 00:55:33,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-14 00:55:33,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-14 00:55:33,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-14 00:55:33,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-14 00:55:33,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-14 00:55:33,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-14 00:55:33,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-14 00:55:33,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-10-14 00:55:33,132 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-14 00:55:33,132 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1032630632:90 , srv1041461872:90 , srv1161953225:90 , srv1169219308:90 , srv1202873223:90 , srv1222893939:90 , srv1227768654:90 , srv131520823:90 , srv1470018454:90 , srv1493857472:90 , srv1591576387:90 , srv1599229116:90 , srv1608814558:90 , srv1810250614:90 , srv1952473604:90 , srv2076048964:90 , srv209025133:90 , srv2119268426:90 , srv2346969:90 , srv344465465:90 , srv409206569:90 , srv536889466:90 , srv552594602:90 , srv554049766:90 , srv64121835:90 , srv717825470:90 , srv77175318:90 , srv784344373:90 , srv907682222:90 , srv909046918:90 }
2023-10-14 00:55:33,150 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=254 (was 340), ProcessCount=171 (was 171), AvailableMemoryMB=2004 (was 1682) - AvailableMemoryMB LEAK? -