2023-12-10 14:53:16,372 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-12-10 14:53:16,700 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-12-10 14:53:16,818 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-12-10 14:53:16,893 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-12-10 14:53:16,903 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-12-10 14:53:16,920 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=317, ProcessCount=171, AvailableMemoryMB=6528
2023-12-10 14:53:16,923 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-12-10 14:53:16,923 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-12-10 14:53:17,061 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv10343398:84 , srv1035967272:84 , srv1061732442:84 , srv1075361303:84 , srv108669995:84 , srv1108166217:84 , srv1133111504:84 , srv1251813174:84 , srv126900030:84 , srv1271784353:84 , srv1278212773:84 , srv1320502795:84 , srv1346851867:84 , srv1354670017:84 , srv1365574232:84 , srv1371357047:84 , srv1374740009:84 , srv1399500287:84 , srv1445448910:84 , srv1496989528:84 , srv1529625402:84 , srv1552266058:84 , srv1559503414:84 , srv1565841915:84 , srv1594093748:84 , srv1603746553:84 , srv160390282:84 , srv1636850184:84 , srv1653786190:84 , srv1663364360:84 , srv1675373821:84 , srv1683015403:84 , srv1700725197:84 , srv1722590683:84 , srv1726034099:84 , srv1779533764:84 , srv1797387661:84 , srv1801401693:84 , srv180205340:84 , srv1817794527:84 , srv1822905643:84 , srv1847947732:84 , srv1850710610:84 , srv1857214932:84 , srv1871454894:84 , srv1910898875:84 , srv191605104:84 , srv1925165518:84 , srv1930397581:84 , srv1953243455:84 , srv19883039:84 , srv2006061771:84 , srv2027412567:84 , srv2040866044:84 , srv2064567425:84 , srv2101671904:84 , srv241799272:84 , srv290252676:84 , srv294793661:84 , srv30542488:84 , srv309511100:84 , srv31826464:84 , srv324142573:84 , srv336788016:84 , srv347825498:84 , srv348910497:84 , srv356321920:84 , srv356790599:84 , srv371623541:84 , srv375852887:84 , srv409143923:84 , srv412729324:84 , srv424617870:84 , srv47223291:84 , srv492040476:84 , srv50113851:84 , srv55825876:84 , srv589077944:84 , srv602668324:84 , srv604105808:84 , srv642638832:84 , srv669730740:84 , srv692452562:84 , srv698229280:84 , srv704737015:84 , srv711457360:84 , srv712501648:84 , srv723443036:84 , srv744397425:84 , srv778294445:84 , srv783346240:84 , srv790805176:84 , srv797664527:84 , srv831972068:84 , srv868147480:84 , srv871541661:84 , srv916658535:84 , srv922110423:84 , srv959930171:84 , srv78574146:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-12-10 14:53:17,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-10 14:53:17,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv371623541=68, srv1594093748=24, srv375852887=69, srv492040476=74, srv1722590683=33, srv797664527=93, srv642638832=80, srv10343398=0, srv1445448910=18, srv1365574232=14, srv30542488=59, srv723443036=87, srv50113851=75, srv744397425=88, srv1910898875=45, srv191605104=46, srv1529625402=20, srv1108166217=5, srv336788016=63, srv1801401693=37, srv78574146=91, srv669730740=81, srv1320502795=11, srv1271784353=9, srv1930397581=48, srv409143923=70, srv2101671904=55, srv1700725197=32, srv1822905643=40, srv1797387661=36, srv704737015=84, srv1133111504=6, srv831972068=94, srv1075361303=3, srv1779533764=35, srv1817794527=39, srv1636850184=27, srv1565841915=23, srv47223291=73, srv356321920=66, srv126900030=8, srv871541661=96, srv712501648=86, srv916658535=97, srv778294445=89, srv1857214932=43, srv108669995=4, srv1061732442=2, srv868147480=95, srv692452562=82, srv424617870=72, srv1653786190=28, srv602668324=78, srv324142573=62, srv348910497=65, srv1675373821=30, srv698229280=83, srv31826464=61, srv412729324=71, srv356790599=67, srv1371357047=15, srv1354670017=13, srv290252676=57, srv790805176=92, srv1847947732=41, srv241799272=56, srv1278212773=10, srv1399500287=17, srv2040866044=53, srv160390282=26, srv19883039=50, srv783346240=90, srv2027412567=52, srv959930171=99, srv1726034099=34, srv1871454894=44, srv1496989528=19, srv1683015403=31, srv711457360=85, srv1552266058=21, srv1925165518=47, srv2006061771=51, srv922110423=98, srv1374740009=16, srv1603746553=25, srv1663364360=29, srv2064567425=54, srv604105808=79, srv1035967272=1, srv347825498=64, srv1850710610=42, srv294793661=58, srv1953243455=49, srv589077944=77, srv1346851867=12, srv1559503414=22, srv309511100=60, srv180205340=38, srv55825876=76, srv1251813174=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-10 14:53:17,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-10 14:53:17,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-10 14:53:17,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-10 14:53:17,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-10 14:53:17,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-10 14:53:17,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-10 14:53:17,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-10 14:53:17,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-10 14:53:17,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-10 14:53:17,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-10 14:53:17,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-10 14:53:17,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-10 14:53:17,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-10 14:53:17,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-10 14:53:17,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-10 14:53:17,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-10 14:53:17,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-10 14:53:17,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-10 14:53:17,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-10 14:53:17,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-10 14:53:17,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-10 14:53:17,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-10 14:53:17,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-10 14:53:17,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-10 14:53:17,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-10 14:53:17,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-10 14:53:17,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-10 14:53:17,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-10 14:53:17,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-10 14:53:17,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-10 14:53:17,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-10 14:53:17,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-10 14:53:17,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-10 14:53:17,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-10 14:53:17,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-10 14:53:17,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-10 14:53:17,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-10 14:53:17,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-10 14:53:17,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-10 14:53:17,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-10 14:53:17,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-10 14:53:17,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-10 14:53:17,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-10 14:53:17,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-10 14:53:17,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-10 14:53:17,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-10 14:53:17,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-10 14:53:17,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-10 14:53:17,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-10 14:53:17,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-10 14:53:17,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-10 14:53:17,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-10 14:53:17,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-10 14:53:17,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-10 14:53:17,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-10 14:53:17,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-10 14:53:17,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-10 14:53:17,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-10 14:53:17,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-10 14:53:17,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-10 14:53:17,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-10 14:53:17,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-10 14:53:17,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-10 14:53:17,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-10 14:53:17,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-10 14:53:17,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-10 14:53:17,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-10 14:53:17,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-10 14:53:17,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-10 14:53:17,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-10 14:53:17,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-10 14:53:17,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-10 14:53:17,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-10 14:53:17,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-10 14:53:17,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-10 14:53:17,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-10 14:53:17,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-10 14:53:17,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-10 14:53:17,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-10 14:53:17,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-10 14:53:17,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-10 14:53:17,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-10 14:53:17,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-10 14:53:17,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-10 14:53:17,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-10 14:53:17,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-10 14:53:17,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-10 14:53:17,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-10 14:53:17,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-10 14:53:17,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-10 14:53:17,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-10 14:53:17,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-10 14:53:17,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-10 14:53:17,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-10 14:53:17,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-10 14:53:17,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-10 14:53:17,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-10 14:53:17,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-10 14:53:17,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-10 14:53:17,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-10 14:53:17,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-10 14:53:17,607 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-10 14:53:17,607 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-12-10 14:53:17,608 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-12-10 14:53:18,060 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 830 ms to try 5000 different iterations.  Found a solution that moves 4726 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17719408732916608. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0379431253087997, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05475791022091753, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5251111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0379431253087997, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17241666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23891666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-10 14:53:18,078 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv2064567425:81 , srv10343398:82 , srv1722590683:82 , srv1726034099:82 , srv1953243455:82 , srv356321920:82 , srv47223291:82 , srv723443036:82 , srv1075361303:83 , srv1552266058:83 , srv1559503414:83 , srv602668324:83 , srv1278212773:84 , srv1399500287:84 , srv1663364360:84 , srv180205340:84 , srv348910497:84 , srv371623541:84 , srv409143923:84 , srv711457360:84 , srv712501648:84 , srv1354670017:85 , srv1565841915:85 , srv1700725197:85 , srv1797387661:85 , srv1822905643:85 , srv1850710610:85 , srv2027412567:85 , srv241799272:85 , srv30542488:85 , srv324142573:85 , srv412729324:85 , srv589077944:85 , srv604105808:85 , srv669730740:85 , srv922110423:85 , srv1035967272:86 , srv126900030:86 , srv1271784353:86 , srv1496989528:86 , srv160390282:86 , srv1683015403:86 , srv1925165518:86 , srv375852887:86 , srv642638832:86 , srv698229280:86 , srv778294445:86 , srv868147480:86 , srv871541661:86 , srv108669995:87 , srv1133111504:87 , srv1365574232:87 , srv1445448910:87 , srv1653786190:87 , srv1857214932:87 , srv1930397581:87 , srv19883039:87 , srv2006061771:87 , srv2040866044:87 , srv294793661:87 , srv309511100:87 , srv744397425:87 , srv783346240:87 , srv790805176:87 , srv1108166217:88 , srv1801401693:88 , srv1817794527:88 , srv1847947732:88 , srv1910898875:88 , srv2101671904:88 , srv31826464:88 , srv336788016:88 , srv356790599:88 , srv55825876:88 , srv797664527:88 , srv916658535:88 , srv959930171:88 , srv1320502795:89 , srv1371357047:89 , srv1374740009:89 , srv1675373821:89 , srv347825498:89 , srv424617870:89 , srv492040476:89 , srv692452562:89 , srv704737015:89 , srv1251813174:90 , srv1594093748:90 , srv1779533764:90 , srv1871454894:90 , srv191605104:90 , srv1346851867:91 , srv1636850184:91 , srv290252676:91 , srv50113851:91 , srv1603746553:92 , srv1061732442:93 , srv831972068:93 , srv1529625402:95 , srv78574146:427 }
2023-12-10 14:53:18,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-10 14:53:18,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv371623541=68, srv1594093748=24, srv375852887=69, srv492040476=74, srv1722590683=33, srv797664527=93, srv642638832=80, srv10343398=0, srv1445448910=18, srv1365574232=14, srv30542488=59, srv723443036=87, srv50113851=75, srv744397425=88, srv1910898875=45, srv191605104=46, srv1529625402=20, srv1108166217=5, srv336788016=63, srv1801401693=37, srv78574146=91, srv669730740=81, srv1320502795=11, srv1271784353=9, srv1930397581=48, srv409143923=70, srv2101671904=55, srv1700725197=32, srv1822905643=40, srv1797387661=36, srv704737015=84, srv1133111504=6, srv831972068=94, srv1075361303=3, srv1779533764=35, srv1817794527=39, srv1636850184=27, srv1565841915=23, srv47223291=73, srv356321920=66, srv126900030=8, srv871541661=96, srv712501648=86, srv916658535=97, srv778294445=89, srv1857214932=43, srv108669995=4, srv1061732442=2, srv868147480=95, srv692452562=82, srv424617870=72, srv1653786190=28, srv602668324=78, srv324142573=62, srv348910497=65, srv1675373821=30, srv698229280=83, srv31826464=61, srv412729324=71, srv356790599=67, srv1371357047=15, srv1354670017=13, srv290252676=57, srv790805176=92, srv1847947732=41, srv241799272=56, srv1278212773=10, srv1399500287=17, srv2040866044=53, srv160390282=26, srv19883039=50, srv783346240=90, srv2027412567=52, srv959930171=99, srv1726034099=34, srv1871454894=44, srv1496989528=19, srv1683015403=31, srv711457360=85, srv1552266058=21, srv1925165518=47, srv2006061771=51, srv922110423=98, srv1374740009=16, srv1603746553=25, srv1663364360=29, srv2064567425=54, srv604105808=79, srv1035967272=1, srv347825498=64, srv1850710610=42, srv294793661=58, srv1953243455=49, srv589077944=77, srv1346851867=12, srv1559503414=22, srv309511100=60, srv180205340=38, srv55825876=76, srv1251813174=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-10 14:53:18,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-10 14:53:18,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-10 14:53:18,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-10 14:53:18,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-10 14:53:18,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-10 14:53:18,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-10 14:53:18,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-10 14:53:18,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-10 14:53:18,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-10 14:53:18,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-10 14:53:18,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-10 14:53:18,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-10 14:53:18,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-10 14:53:18,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-10 14:53:18,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-10 14:53:18,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-10 14:53:18,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-10 14:53:18,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-10 14:53:18,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-10 14:53:18,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-10 14:53:18,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-10 14:53:18,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-10 14:53:18,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-10 14:53:18,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-10 14:53:18,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-10 14:53:18,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-10 14:53:18,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-10 14:53:18,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-10 14:53:18,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-10 14:53:18,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-10 14:53:18,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-10 14:53:18,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-10 14:53:18,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-10 14:53:18,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-10 14:53:18,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-10 14:53:18,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-10 14:53:18,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-10 14:53:18,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-10 14:53:18,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-10 14:53:18,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-10 14:53:18,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-10 14:53:18,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-10 14:53:18,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-10 14:53:18,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-10 14:53:18,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-10 14:53:18,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-10 14:53:18,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-10 14:53:18,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-10 14:53:18,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-10 14:53:18,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-10 14:53:18,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-10 14:53:18,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-10 14:53:18,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-10 14:53:18,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-10 14:53:18,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-10 14:53:18,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-10 14:53:18,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-10 14:53:18,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-10 14:53:18,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-10 14:53:18,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-10 14:53:18,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-10 14:53:18,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-10 14:53:18,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-10 14:53:18,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-10 14:53:18,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-10 14:53:18,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-10 14:53:18,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-10 14:53:18,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-10 14:53:18,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-10 14:53:18,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-10 14:53:18,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-10 14:53:18,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-10 14:53:18,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-10 14:53:18,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-10 14:53:18,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-10 14:53:18,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-10 14:53:18,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-10 14:53:18,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-10 14:53:18,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-10 14:53:18,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-10 14:53:18,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-10 14:53:18,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-10 14:53:18,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-10 14:53:18,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-10 14:53:18,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-10 14:53:18,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-10 14:53:18,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-10 14:53:18,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-10 14:53:18,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-10 14:53:18,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-10 14:53:18,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-10 14:53:18,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-10 14:53:18,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-10 14:53:18,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-10 14:53:18,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-10 14:53:18,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-10 14:53:18,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-10 14:53:18,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-10 14:53:18,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-10 14:53:18,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-10 14:53:18,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-10 14:53:18,298 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0379431253087997, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05475791022091753, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0379431253087997, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17241666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23891666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-10 14:53:18,298 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-12-10 14:53:18,298 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.1771609951698732, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0379431253087997, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05475791022091753, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0379431253087997, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17241666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23891666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-12-10 14:53:18,607 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 451 ms to try 5000 different iterations.  Found a solution that moves 2635 regions; Going from a computed imbalance of 0.1771609951698732 to a new imbalance of 0.015059195031412296. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02029954015710329); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.034641016151377546, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2927777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02029954015710329); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011666666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04758333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-10 14:53:18,616 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1529625402:77 , srv290252676:79 , srv1346851867:80 , srv1061732442:81 , srv1603746553:81 , srv1779533764:81 , srv1871454894:81 , srv704737015:81 , srv831972068:81 , srv1374740009:82 , srv1675373821:82 , srv1910898875:82 , srv50113851:82 , srv1108166217:83 , srv1251813174:83 , srv1320502795:83 , srv1365574232:83 , srv1371357047:83 , srv1445448910:83 , srv1801401693:83 , srv1930397581:83 , srv347825498:83 , srv492040476:83 , srv589077944:83 , srv692452562:83 , srv797664527:83 , srv959930171:83 , srv1133111504:84 , srv1636850184:84 , srv1822905643:84 , srv1857214932:84 , srv2101671904:84 , srv336788016:84 , srv356790599:84 , srv424617870:84 , srv55825876:84 , srv1271784353:85 , srv1496989528:85 , srv1594093748:85 , srv160390282:85 , srv1700725197:85 , srv191605104:85 , srv19883039:85 , srv2027412567:85 , srv2040866044:85 , srv294793661:85 , srv309511100:85 , srv31826464:85 , srv642638832:85 , srv669730740:85 , srv790805176:85 , srv916658535:85 , srv922110423:85 , srv1565841915:86 , srv30542488:86 , srv348910497:86 , srv412729324:86 , srv698229280:86 , srv744397425:86 , srv783346240:86 , srv871541661:86 , srv126900030:87 , srv1552266058:87 , srv1683015403:87 , srv1817794527:87 , srv1847947732:87 , srv1850710610:87 , srv2006061771:87 , srv241799272:87 , srv375852887:87 , srv868147480:87 , srv1035967272:88 , srv108669995:88 , srv1278212773:88 , srv1399500287:88 , srv1559503414:88 , srv1726034099:88 , srv1797387661:88 , srv371623541:88 , srv604105808:88 , srv712501648:88 , srv1075361303:89 , srv1354670017:89 , srv1663364360:89 , srv1722590683:89 , srv180205340:89 , srv409143923:89 , srv47223291:89 , srv602668324:89 , srv711457360:89 , srv778294445:89 , srv1925165518:90 , srv324142573:90 , srv356321920:90 , srv10343398:91 , srv2064567425:92 , srv723443036:92 , srv1953243455:93 , srv1653786190:94 , srv78574146:527 }
2023-12-10 14:53:18,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-10 14:53:18,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv371623541=68, srv1594093748=24, srv375852887=69, srv492040476=74, srv1722590683=33, srv797664527=93, srv642638832=80, srv10343398=0, srv1445448910=18, srv1365574232=14, srv30542488=59, srv723443036=87, srv50113851=75, srv744397425=88, srv1910898875=45, srv191605104=46, srv1529625402=20, srv1108166217=5, srv336788016=63, srv1801401693=37, srv78574146=91, srv669730740=81, srv1320502795=11, srv1271784353=9, srv1930397581=48, srv409143923=70, srv2101671904=55, srv1700725197=32, srv1822905643=40, srv1797387661=36, srv704737015=84, srv1133111504=6, srv831972068=94, srv1075361303=3, srv1779533764=35, srv1817794527=39, srv1636850184=27, srv1565841915=23, srv47223291=73, srv356321920=66, srv126900030=8, srv871541661=96, srv712501648=86, srv916658535=97, srv778294445=89, srv1857214932=43, srv108669995=4, srv1061732442=2, srv868147480=95, srv692452562=82, srv424617870=72, srv1653786190=28, srv602668324=78, srv324142573=62, srv348910497=65, srv1675373821=30, srv698229280=83, srv31826464=61, srv412729324=71, srv356790599=67, srv1371357047=15, srv1354670017=13, srv290252676=57, srv790805176=92, srv1847947732=41, srv241799272=56, srv1278212773=10, srv1399500287=17, srv2040866044=53, srv160390282=26, srv19883039=50, srv783346240=90, srv2027412567=52, srv959930171=99, srv1726034099=34, srv1871454894=44, srv1496989528=19, srv1683015403=31, srv711457360=85, srv1552266058=21, srv1925165518=47, srv2006061771=51, srv922110423=98, srv1374740009=16, srv1603746553=25, srv1663364360=29, srv2064567425=54, srv604105808=79, srv1035967272=1, srv347825498=64, srv1850710610=42, srv294793661=58, srv1953243455=49, srv589077944=77, srv1346851867=12, srv1559503414=22, srv309511100=60, srv180205340=38, srv55825876=76, srv1251813174=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-10 14:53:18,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-10 14:53:18,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-10 14:53:18,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-10 14:53:18,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-10 14:53:18,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-10 14:53:18,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-10 14:53:18,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-10 14:53:18,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-10 14:53:18,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-10 14:53:18,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-10 14:53:18,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-10 14:53:18,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-10 14:53:18,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-10 14:53:18,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-10 14:53:18,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-10 14:53:18,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-10 14:53:18,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-10 14:53:18,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-10 14:53:18,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-10 14:53:18,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-10 14:53:18,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-10 14:53:18,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-10 14:53:18,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-10 14:53:18,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-10 14:53:18,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-10 14:53:18,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-10 14:53:18,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-10 14:53:18,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-10 14:53:18,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-10 14:53:18,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-10 14:53:18,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-10 14:53:18,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-10 14:53:18,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-10 14:53:18,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-10 14:53:18,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-10 14:53:18,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-10 14:53:18,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-10 14:53:18,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-10 14:53:18,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-10 14:53:18,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-10 14:53:18,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-10 14:53:18,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-10 14:53:18,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-10 14:53:18,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-10 14:53:18,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-10 14:53:18,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-10 14:53:18,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-10 14:53:18,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-10 14:53:18,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-10 14:53:18,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-10 14:53:18,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-10 14:53:18,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-10 14:53:18,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-10 14:53:18,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-10 14:53:18,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-10 14:53:18,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-10 14:53:18,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-10 14:53:18,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-10 14:53:18,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-10 14:53:18,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-10 14:53:18,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-10 14:53:18,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-10 14:53:18,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-10 14:53:18,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-10 14:53:18,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-10 14:53:18,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-10 14:53:18,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-10 14:53:18,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-10 14:53:18,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-10 14:53:18,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-10 14:53:18,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-10 14:53:18,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-10 14:53:18,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-10 14:53:18,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-10 14:53:18,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-10 14:53:18,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-10 14:53:18,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-10 14:53:18,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-10 14:53:18,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-10 14:53:18,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-10 14:53:18,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-10 14:53:18,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-10 14:53:18,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-10 14:53:18,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-10 14:53:18,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-10 14:53:18,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-10 14:53:18,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-10 14:53:18,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-10 14:53:18,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-10 14:53:18,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-10 14:53:18,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-10 14:53:18,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-10 14:53:18,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-10 14:53:18,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-10 14:53:18,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-10 14:53:18,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-10 14:53:18,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-10 14:53:18,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-10 14:53:18,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-10 14:53:18,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-10 14:53:18,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-10 14:53:18,770 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02029954015710329); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.034641016151377546, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02029954015710329); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011666666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04758333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-10 14:53:18,770 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-12-10 14:53:18,770 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.015040744367058339, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02029954015710329); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.034641016151377546, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02029954015710329); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011666666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04758333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-12-10 14:53:19,065 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 404 ms to try 5000 different iterations.  Found a solution that moves 1033 regions; Going from a computed imbalance of 0.015040744367058339 to a new imbalance of 6.333209166156051E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0059908064411803305); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009344389795537962); MoveCostFunction : (multiplier=7.0, imbalance=0.11477777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0059908064411803305); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0045); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-10 14:53:19,066 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1653786190:77 , srv1594093748:82 , srv1636850184:82 , srv1925165518:82 , srv1953243455:82 , srv1035967272:83 , srv108669995:83 , srv1354670017:83 , srv1817794527:83 , srv1847947732:83 , srv191605104:83 , srv2006061771:83 , srv324142573:83 , srv10343398:84 , srv1061732442:84 , srv1251813174:84 , srv126900030:84 , srv1663364360:84 , srv1683015403:84 , srv1797387661:84 , srv180205340:84 , srv2064567425:84 , srv31826464:84 , srv409143923:84 , srv424617870:84 , srv50113851:84 , srv604105808:84 , srv698229280:84 , srv723443036:84 , srv744397425:84 , srv778294445:84 , srv1075361303:85 , srv1320502795:85 , srv1529625402:85 , srv1603746553:85 , srv1675373821:85 , srv1850710610:85 , srv2101671904:85 , srv241799272:85 , srv356321920:85 , srv356790599:85 , srv375852887:85 , srv492040476:85 , srv602668324:85 , srv711457360:85 , srv783346240:85 , srv790805176:85 , srv831972068:85 , srv1133111504:86 , srv1278212773:86 , srv1371357047:86 , srv1374740009:86 , srv1399500287:86 , srv1496989528:86 , srv1552266058:86 , srv1559503414:86 , srv1565841915:86 , srv160390282:86 , srv1722590683:86 , srv1779533764:86 , srv1801401693:86 , srv19883039:86 , srv2040866044:86 , srv294793661:86 , srv30542488:86 , srv309511100:86 , srv336788016:86 , srv347825498:86 , srv371623541:86 , srv412729324:86 , srv55825876:86 , srv642638832:86 , srv692452562:86 , srv712501648:86 , srv868147480:86 , srv871541661:86 , srv916658535:86 , srv959930171:86 , srv1108166217:87 , srv1271784353:87 , srv1346851867:87 , srv1700725197:87 , srv1857214932:87 , srv1871454894:87 , srv1910898875:87 , srv1930397581:87 , srv2027412567:87 , srv348910497:87 , srv47223291:87 , srv669730740:87 , srv704737015:87 , srv797664527:87 , srv1365574232:88 , srv1445448910:88 , srv1726034099:88 , srv1822905643:88 , srv290252676:88 , srv922110423:88 , srv589077944:91 , srv78574146:557 }
2023-12-10 14:53:19,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-10 14:53:19,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv371623541=68, srv1594093748=24, srv375852887=69, srv492040476=74, srv1722590683=33, srv797664527=93, srv642638832=80, srv10343398=0, srv1445448910=18, srv1365574232=14, srv30542488=59, srv723443036=87, srv50113851=75, srv744397425=88, srv1910898875=45, srv191605104=46, srv1529625402=20, srv1108166217=5, srv336788016=63, srv1801401693=37, srv78574146=91, srv669730740=81, srv1320502795=11, srv1271784353=9, srv1930397581=48, srv409143923=70, srv2101671904=55, srv1700725197=32, srv1822905643=40, srv1797387661=36, srv704737015=84, srv1133111504=6, srv831972068=94, srv1075361303=3, srv1779533764=35, srv1817794527=39, srv1636850184=27, srv1565841915=23, srv47223291=73, srv356321920=66, srv126900030=8, srv871541661=96, srv712501648=86, srv916658535=97, srv778294445=89, srv1857214932=43, srv108669995=4, srv1061732442=2, srv868147480=95, srv692452562=82, srv424617870=72, srv1653786190=28, srv602668324=78, srv324142573=62, srv348910497=65, srv1675373821=30, srv698229280=83, srv31826464=61, srv412729324=71, srv356790599=67, srv1371357047=15, srv1354670017=13, srv290252676=57, srv790805176=92, srv1847947732=41, srv241799272=56, srv1278212773=10, srv1399500287=17, srv2040866044=53, srv160390282=26, srv19883039=50, srv783346240=90, srv2027412567=52, srv959930171=99, srv1726034099=34, srv1871454894=44, srv1496989528=19, srv1683015403=31, srv711457360=85, srv1552266058=21, srv1925165518=47, srv2006061771=51, srv922110423=98, srv1374740009=16, srv1603746553=25, srv1663364360=29, srv2064567425=54, srv604105808=79, srv1035967272=1, srv347825498=64, srv1850710610=42, srv294793661=58, srv1953243455=49, srv589077944=77, srv1346851867=12, srv1559503414=22, srv309511100=60, srv180205340=38, srv55825876=76, srv1251813174=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-10 14:53:19,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-10 14:53:19,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-10 14:53:19,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-10 14:53:19,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-10 14:53:19,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-10 14:53:19,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-10 14:53:19,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-10 14:53:19,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-10 14:53:19,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-10 14:53:19,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-10 14:53:19,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-10 14:53:19,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-10 14:53:19,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-10 14:53:19,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-10 14:53:19,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-10 14:53:19,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-10 14:53:19,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-10 14:53:19,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-10 14:53:19,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-10 14:53:19,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-10 14:53:19,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-10 14:53:19,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-10 14:53:19,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-10 14:53:19,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-10 14:53:19,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-10 14:53:19,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-10 14:53:19,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-10 14:53:19,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-10 14:53:19,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-10 14:53:19,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-10 14:53:19,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-10 14:53:19,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-10 14:53:19,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-10 14:53:19,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-10 14:53:19,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-10 14:53:19,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-10 14:53:19,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-10 14:53:19,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-10 14:53:19,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-10 14:53:19,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-10 14:53:19,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-10 14:53:19,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-10 14:53:19,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-10 14:53:19,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-10 14:53:19,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-10 14:53:19,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-10 14:53:19,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-10 14:53:19,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-10 14:53:19,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-10 14:53:19,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-10 14:53:19,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-10 14:53:19,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-10 14:53:19,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-10 14:53:19,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-10 14:53:19,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-10 14:53:19,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-10 14:53:19,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-10 14:53:19,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-10 14:53:19,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-10 14:53:19,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-10 14:53:19,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-10 14:53:19,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-10 14:53:19,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-10 14:53:19,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-10 14:53:19,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-10 14:53:19,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-10 14:53:19,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-10 14:53:19,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-10 14:53:19,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-10 14:53:19,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-10 14:53:19,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-10 14:53:19,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-10 14:53:19,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-10 14:53:19,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-10 14:53:19,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-10 14:53:19,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-10 14:53:19,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-10 14:53:19,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-10 14:53:19,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-10 14:53:19,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-10 14:53:19,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-10 14:53:19,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-10 14:53:19,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-10 14:53:19,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-10 14:53:19,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-10 14:53:19,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-10 14:53:19,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-10 14:53:19,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-10 14:53:19,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-10 14:53:19,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-10 14:53:19,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-10 14:53:19,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-10 14:53:19,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-10 14:53:19,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-10 14:53:19,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-10 14:53:19,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-10 14:53:19,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-10 14:53:19,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-10 14:53:19,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-10 14:53:19,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-10 14:53:19,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-10 14:53:19,230 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0059908064411803305); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009344389795537962); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0059908064411803305); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0045); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-10 14:53:19,230 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-12-10 14:53:19,230 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=6.260876960168814E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0059908064411803305); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009344389795537962); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0059908064411803305); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0045); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-12-10 14:53:19,664 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 563 ms to try 5000 different iterations.  Found a solution that moves 289 regions; Going from a computed imbalance of 6.260876960168814E-4 to a new imbalance of 2.1821797687305966E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.23782800878924E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0021711298242630987); MoveCostFunction : (multiplier=7.0, imbalance=0.03211111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.23782800878924E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-10 14:53:19,665 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1365574232:83 , srv1371357047:83 , srv1726034099:83 , srv1925165518:83 , srv589077944:83 , srv868147480:83 , srv916658535:83 , srv1061732442:84 , srv1075361303:84 , srv108669995:84 , srv1271784353:84 , srv1346851867:84 , srv1399500287:84 , srv1445448910:84 , srv1496989528:84 , srv1565841915:84 , srv1603746553:84 , srv1653786190:84 , srv1683015403:84 , srv1779533764:84 , srv1817794527:84 , srv1847947732:84 , srv1857214932:84 , srv1871454894:84 , srv191605104:84 , srv2040866044:84 , srv290252676:84 , srv294793661:84 , srv309511100:84 , srv31826464:84 , srv324142573:84 , srv336788016:84 , srv347825498:84 , srv371623541:84 , srv375852887:84 , srv47223291:84 , srv50113851:84 , srv55825876:84 , srv692452562:84 , srv711457360:84 , srv712501648:84 , srv723443036:84 , srv778294445:84 , srv783346240:84 , srv797664527:84 , srv831972068:84 , srv871541661:84 , srv922110423:84 , srv10343398:85 , srv1035967272:85 , srv1108166217:85 , srv1133111504:85 , srv1251813174:85 , srv1278212773:85 , srv1320502795:85 , srv1354670017:85 , srv1374740009:85 , srv1529625402:85 , srv1559503414:85 , srv1594093748:85 , srv160390282:85 , srv1636850184:85 , srv1663364360:85 , srv1700725197:85 , srv1722590683:85 , srv1797387661:85 , srv1801401693:85 , srv1822905643:85 , srv1910898875:85 , srv1930397581:85 , srv1953243455:85 , srv19883039:85 , srv2006061771:85 , srv2027412567:85 , srv2064567425:85 , srv2101671904:85 , srv241799272:85 , srv348910497:85 , srv356321920:85 , srv356790599:85 , srv409143923:85 , srv412729324:85 , srv424617870:85 , srv492040476:85 , srv602668324:85 , srv604105808:85 , srv642638832:85 , srv704737015:85 , srv790805176:85 , srv959930171:85 , srv126900030:86 , srv1552266058:86 , srv1675373821:86 , srv180205340:86 , srv1850710610:86 , srv30542488:86 , srv669730740:86 , srv698229280:86 , srv744397425:86 , srv78574146:631 }
2023-12-10 14:53:19,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-10 14:53:19,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv371623541=68, srv1594093748=24, srv375852887=69, srv492040476=74, srv1722590683=33, srv797664527=93, srv642638832=80, srv10343398=0, srv1445448910=18, srv1365574232=14, srv30542488=59, srv723443036=87, srv50113851=75, srv744397425=88, srv1910898875=45, srv191605104=46, srv1529625402=20, srv1108166217=5, srv336788016=63, srv1801401693=37, srv78574146=91, srv669730740=81, srv1320502795=11, srv1271784353=9, srv1930397581=48, srv409143923=70, srv2101671904=55, srv1700725197=32, srv1822905643=40, srv1797387661=36, srv704737015=84, srv1133111504=6, srv831972068=94, srv1075361303=3, srv1779533764=35, srv1817794527=39, srv1636850184=27, srv1565841915=23, srv47223291=73, srv356321920=66, srv126900030=8, srv871541661=96, srv712501648=86, srv916658535=97, srv778294445=89, srv1857214932=43, srv108669995=4, srv1061732442=2, srv868147480=95, srv692452562=82, srv424617870=72, srv1653786190=28, srv602668324=78, srv324142573=62, srv348910497=65, srv1675373821=30, srv698229280=83, srv31826464=61, srv412729324=71, srv356790599=67, srv1371357047=15, srv1354670017=13, srv290252676=57, srv790805176=92, srv1847947732=41, srv241799272=56, srv1278212773=10, srv1399500287=17, srv2040866044=53, srv160390282=26, srv19883039=50, srv783346240=90, srv2027412567=52, srv959930171=99, srv1726034099=34, srv1871454894=44, srv1496989528=19, srv1683015403=31, srv711457360=85, srv1552266058=21, srv1925165518=47, srv2006061771=51, srv922110423=98, srv1374740009=16, srv1603746553=25, srv1663364360=29, srv2064567425=54, srv604105808=79, srv1035967272=1, srv347825498=64, srv1850710610=42, srv294793661=58, srv1953243455=49, srv589077944=77, srv1346851867=12, srv1559503414=22, srv309511100=60, srv180205340=38, srv55825876=76, srv1251813174=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-10 14:53:19,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-10 14:53:19,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-10 14:53:19,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-10 14:53:19,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-10 14:53:19,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-10 14:53:19,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-10 14:53:19,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-10 14:53:19,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-10 14:53:19,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-10 14:53:19,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-10 14:53:19,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-10 14:53:19,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-10 14:53:19,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-10 14:53:19,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-10 14:53:19,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-10 14:53:19,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-10 14:53:19,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-10 14:53:19,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-10 14:53:19,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-10 14:53:19,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-10 14:53:19,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-10 14:53:19,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-10 14:53:19,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-10 14:53:19,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-10 14:53:19,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-10 14:53:19,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-10 14:53:19,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-10 14:53:19,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-10 14:53:19,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-10 14:53:19,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-10 14:53:19,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-10 14:53:19,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-10 14:53:19,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-10 14:53:19,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-10 14:53:19,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-10 14:53:19,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-10 14:53:19,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-10 14:53:19,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-10 14:53:19,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-10 14:53:19,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-10 14:53:19,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-10 14:53:19,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-10 14:53:19,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-10 14:53:19,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-10 14:53:19,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-10 14:53:19,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-10 14:53:19,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-10 14:53:19,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-10 14:53:19,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-10 14:53:19,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-10 14:53:19,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-10 14:53:19,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-10 14:53:19,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-10 14:53:19,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-10 14:53:19,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-10 14:53:19,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-10 14:53:19,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-10 14:53:19,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-10 14:53:19,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-10 14:53:19,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-10 14:53:19,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-10 14:53:19,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-10 14:53:19,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-10 14:53:19,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-10 14:53:19,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-10 14:53:19,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-10 14:53:19,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-10 14:53:19,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-10 14:53:19,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-10 14:53:19,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-10 14:53:19,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-10 14:53:19,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-10 14:53:19,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-10 14:53:19,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-10 14:53:19,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-10 14:53:19,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-10 14:53:19,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-10 14:53:19,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-10 14:53:19,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-10 14:53:19,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-10 14:53:19,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-10 14:53:19,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-10 14:53:19,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-10 14:53:19,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-10 14:53:19,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-10 14:53:19,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-10 14:53:19,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-10 14:53:19,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-10 14:53:19,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-10 14:53:19,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-10 14:53:19,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-10 14:53:19,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-10 14:53:19,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-10 14:53:19,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-10 14:53:19,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-10 14:53:19,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-10 14:53:19,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-10 14:53:19,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-10 14:53:19,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-10 14:53:19,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-10 14:53:19,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-10 14:53:19,825 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.9798176435581686E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.23782800878924E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0021711298242630987); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.23782800878924E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-10 14:53:19,826 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1365574232:83 , srv1371357047:83 , srv1726034099:83 , srv1925165518:83 , srv589077944:83 , srv868147480:83 , srv916658535:83 , srv1061732442:84 , srv1075361303:84 , srv108669995:84 , srv1271784353:84 , srv1346851867:84 , srv1399500287:84 , srv1445448910:84 , srv1496989528:84 , srv1565841915:84 , srv1603746553:84 , srv1653786190:84 , srv1683015403:84 , srv1779533764:84 , srv1817794527:84 , srv1847947732:84 , srv1857214932:84 , srv1871454894:84 , srv191605104:84 , srv2040866044:84 , srv290252676:84 , srv294793661:84 , srv309511100:84 , srv31826464:84 , srv324142573:84 , srv336788016:84 , srv347825498:84 , srv371623541:84 , srv375852887:84 , srv47223291:84 , srv50113851:84 , srv55825876:84 , srv692452562:84 , srv711457360:84 , srv712501648:84 , srv723443036:84 , srv778294445:84 , srv783346240:84 , srv797664527:84 , srv831972068:84 , srv871541661:84 , srv922110423:84 , srv10343398:85 , srv1035967272:85 , srv1108166217:85 , srv1133111504:85 , srv1251813174:85 , srv1278212773:85 , srv1320502795:85 , srv1354670017:85 , srv1374740009:85 , srv1529625402:85 , srv1559503414:85 , srv1594093748:85 , srv160390282:85 , srv1636850184:85 , srv1663364360:85 , srv1700725197:85 , srv1722590683:85 , srv1797387661:85 , srv1801401693:85 , srv1822905643:85 , srv1910898875:85 , srv1930397581:85 , srv1953243455:85 , srv19883039:85 , srv2006061771:85 , srv2027412567:85 , srv2064567425:85 , srv2101671904:85 , srv241799272:85 , srv348910497:85 , srv356321920:85 , srv356790599:85 , srv409143923:85 , srv412729324:85 , srv424617870:85 , srv492040476:85 , srv602668324:85 , srv604105808:85 , srv642638832:85 , srv704737015:85 , srv790805176:85 , srv959930171:85 , srv126900030:86 , srv1552266058:86 , srv1675373821:86 , srv180205340:86 , srv1850710610:86 , srv30542488:86 , srv669730740:86 , srv698229280:86 , srv744397425:86 , srv78574146:631 }
2023-12-10 14:53:19,867 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=356 (was 317) - SystemLoadAverage LEAK? -, ProcessCount=171 (was 171), AvailableMemoryMB=5653 (was 6528)
2023-12-10 14:53:19,878 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=356, ProcessCount=171, AvailableMemoryMB=5650
2023-12-10 14:53:19,879 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-12-10 14:53:19,879 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-12-10 14:53:19,894 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1092363605:84 , srv1111290466:84 , srv1112770532:84 , srv1157652382:84 , srv1224008585:84 , srv1259348552:84 , srv1262272409:84 , srv1289174097:84 , srv1304643359:84 , srv1338821784:84 , srv1412905040:84 , srv1477236212:84 , srv1489387272:84 , srv1517931718:84 , srv1796108507:84 , srv1828520043:84 , srv2026018080:84 , srv307979945:84 , srv335464917:84 , srv342542019:84 , srv572938027:84 , srv624847004:84 , srv745187505:84 , srv745853492:84 , srv758293152:84 , srv935363592:84 , srv936171252:84 , srv958162058:84 , srv983033902:84 , srv831419665:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-12-10 14:53:19,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-10 14:53:19,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1412905040=10, srv1092363605=0, srv1111290466=1, srv1828520043=15, srv1289174097=7, srv624847004=21, srv936171252=27, srv745187505=22, srv1262272409=6, srv307979945=17, srv1259348552=5, srv335464917=18, srv1224008585=4, srv983033902=29, srv1489387272=12, srv2026018080=16, srv342542019=19, srv1338821784=9, srv1796108507=14, srv1112770532=2, srv958162058=28, srv1477236212=11, srv745853492=23, srv1157652382=3, srv831419665=25, srv1304643359=8, srv935363592=26, srv1517931718=13, srv758293152=24, srv572938027=20} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-10 14:53:19,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-10 14:53:19,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-10 14:53:19,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-10 14:53:19,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-10 14:53:19,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-10 14:53:19,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-10 14:53:19,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-10 14:53:19,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-10 14:53:19,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-10 14:53:19,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-10 14:53:19,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-10 14:53:19,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-10 14:53:19,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-10 14:53:19,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-10 14:53:19,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-10 14:53:19,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-10 14:53:19,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-10 14:53:19,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-10 14:53:19,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-10 14:53:19,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-10 14:53:19,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-10 14:53:19,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-10 14:53:19,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-10 14:53:19,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-10 14:53:19,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-10 14:53:19,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-10 14:53:19,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-10 14:53:19,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-10 14:53:19,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-10 14:53:19,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-10 14:53:19,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-12-10 14:53:19,941 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1768409456405597, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-10 14:53:19,941 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912750263309604, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1768409456405597, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-12-10 14:55:19,918 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5462902 different iterations.  Found a solution that moves 1872 regions; Going from a computed imbalance of 0.9912750263309604 to a new imbalance of 4.3693413878060563E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6933333333333334, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-10 14:55:19,925 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1092363605:90 , srv1111290466:90 , srv1112770532:90 , srv1157652382:90 , srv1224008585:90 , srv1259348552:90 , srv1262272409:90 , srv1289174097:90 , srv1304643359:90 , srv1338821784:90 , srv1412905040:90 , srv1477236212:90 , srv1489387272:90 , srv1517931718:90 , srv1796108507:90 , srv1828520043:90 , srv2026018080:90 , srv307979945:90 , srv335464917:90 , srv342542019:90 , srv572938027:90 , srv624847004:90 , srv745187505:90 , srv745853492:90 , srv758293152:90 , srv831419665:90 , srv935363592:90 , srv936171252:90 , srv958162058:90 , srv983033902:90 }
2023-12-10 14:55:19,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-10 14:55:19,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1412905040=10, srv1092363605=0, srv1111290466=1, srv1828520043=15, srv1289174097=7, srv624847004=21, srv936171252=27, srv745187505=22, srv1262272409=6, srv307979945=17, srv1259348552=5, srv335464917=18, srv1224008585=4, srv983033902=29, srv1489387272=12, srv2026018080=16, srv342542019=19, srv1338821784=9, srv1796108507=14, srv1112770532=2, srv958162058=28, srv1477236212=11, srv745853492=23, srv1157652382=3, srv831419665=25, srv1304643359=8, srv935363592=26, srv1517931718=13, srv758293152=24, srv572938027=20} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-10 14:55:19,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-10 14:55:19,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-10 14:55:19,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-10 14:55:19,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-10 14:55:19,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-10 14:55:19,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-10 14:55:19,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-10 14:55:19,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-10 14:55:19,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-10 14:55:19,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-10 14:55:19,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-10 14:55:19,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-10 14:55:19,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-10 14:55:19,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-10 14:55:19,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-10 14:55:19,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-10 14:55:19,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-10 14:55:19,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-10 14:55:19,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-10 14:55:19,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-10 14:55:19,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-10 14:55:19,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-10 14:55:19,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-10 14:55:19,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-10 14:55:19,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-10 14:55:19,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-10 14:55:19,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-10 14:55:19,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-10 14:55:19,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-10 14:55:19,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-10 14:55:19,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-12-10 14:55:19,946 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-10 14:55:19,947 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1092363605:90 , srv1111290466:90 , srv1112770532:90 , srv1157652382:90 , srv1224008585:90 , srv1259348552:90 , srv1262272409:90 , srv1289174097:90 , srv1304643359:90 , srv1338821784:90 , srv1412905040:90 , srv1477236212:90 , srv1489387272:90 , srv1517931718:90 , srv1796108507:90 , srv1828520043:90 , srv2026018080:90 , srv307979945:90 , srv335464917:90 , srv342542019:90 , srv572938027:90 , srv624847004:90 , srv745187505:90 , srv745853492:90 , srv758293152:90 , srv831419665:90 , srv935363592:90 , srv936171252:90 , srv958162058:90 , srv983033902:90 }
2023-12-10 14:55:19,959 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=270 (was 356), ProcessCount=168 (was 171), AvailableMemoryMB=5222 (was 5650)