2023-08-18 05:53:22,763 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-08-18 05:53:23,080 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-08-18 05:53:23,186 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-08-18 05:53:23,251 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-18 05:53:23,258 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-08-18 05:53:23,271 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=220, ProcessCount=176, AvailableMemoryMB=3768
2023-08-18 05:53:23,273 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-18 05:53:23,274 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-08-18 05:53:23,427 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1037359848:84 , srv1059933114:84 , srv1065312256:84 , srv1068121823:84 , srv1073304041:84 , srv1082822503:84 , srv108474473:84 , srv1097859539:84 , srv1125722661:84 , srv1126083564:84 , srv1171136028:84 , srv1188445380:84 , srv1205472562:84 , srv1211933749:84 , srv12200624:84 , srv1302589586:84 , srv1316185371:84 , srv1324369389:84 , srv1332472545:84 , srv1334277491:84 , srv1391103828:84 , srv1396081822:84 , srv1403876356:84 , srv1414314498:84 , srv1419940182:84 , srv1439452758:84 , srv148886458:84 , srv1489115960:84 , srv1541837401:84 , srv1547802057:84 , srv1582282601:84 , srv1607140967:84 , srv1643700617:84 , srv1665556060:84 , srv1703543222:84 , srv1769354690:84 , srv1798407439:84 , srv1817548068:84 , srv1837060085:84 , srv1840046500:84 , srv1844974711:84 , srv1862930635:84 , srv1865050520:84 , srv187094629:84 , srv190520522:84 , srv1917828270:84 , srv1918749303:84 , srv1927644295:84 , srv1945161511:84 , srv1948110338:84 , srv1968827491:84 , srv1981090411:84 , srv1985270155:84 , srv1997529226:84 , srv1998588546:84 , srv2001536860:84 , srv2001704790:84 , srv2040979279:84 , srv2075697194:84 , srv2079422133:84 , srv2114989359:84 , srv2127410141:84 , srv2128688614:84 , srv2143572639:84 , srv229070811:84 , srv258804845:84 , srv327058812:84 , srv33770308:84 , srv367093400:84 , srv369224354:84 , srv376243884:84 , srv393258092:84 , srv434508687:84 , srv441955381:84 , srv443972406:84 , srv48466051:84 , srv563494468:84 , srv617226982:84 , srv722358233:84 , srv729410354:84 , srv740428352:84 , srv78668300:84 , srv791457266:84 , srv80432903:84 , srv812053759:84 , srv840878023:84 , srv848748801:84 , srv861664772:84 , srv875114548:84 , srv87545706:84 , srv879047950:84 , srv913367377:84 , srv917107262:84 , srv936577314:84 , srv938368361:84 , srv944946571:84 , srv987441101:84 , srv990233025:84 , srv99945518:84 , srv1285867092:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-08-18 05:53:23,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-18 05:53:23,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv148886458=27, srv1665556060=34, srv2114989359=61, srv1862930635=42, srv2040979279=58, srv1285867092=15, srv1865050520=43, srv2128688614=63, srv443972406=75, srv33770308=68, srv369224354=70, srv1419940182=25, srv1582282601=31, srv791457266=83, srv1188445380=11, srv48466051=76, srv1414314498=24, srv190520522=45, srv740428352=81, srv1059933114=1, srv12200624=14, srv840878023=86, srv376243884=71, srv1073304041=4, srv1334277491=20, srv1065312256=2, srv1205472562=12, srv1125722661=8, srv990233025=98, srv1037359848=0, srv1489115960=28, srv1211933749=13, srv258804845=66, srv936577314=94, srv848748801=87, srv2001536860=56, srv944946571=96, srv434508687=73, srv1981090411=52, srv1126083564=9, srv1998588546=55, srv1607140967=32, srv441955381=74, srv80432903=84, srv1324369389=18, srv1068121823=3, srv1837060085=39, srv2075697194=59, srv812053759=85, srv879047950=91, srv1927644295=48, srv1948110338=50, srv1097859539=7, srv108474473=6, srv875114548=89, srv1844974711=41, srv2001704790=57, srv917107262=93, srv563494468=77, srv1968827491=51, srv1997529226=54, srv1798407439=37, srv987441101=97, srv1403876356=23, srv617226982=78, srv1769354690=36, srv722358233=79, srv913367377=92, srv938368361=95, srv1643700617=33, srv1918749303=47, srv1082822503=5, srv1817548068=38, srv1396081822=22, srv1703543222=35, srv1332472545=19, srv861664772=88, srv327058812=67, srv1945161511=49, srv1985270155=53, srv1541837401=29, srv1171136028=10, srv1316185371=17, srv1391103828=21, srv1439452758=26, srv187094629=44, srv78668300=82, srv99945518=99, srv87545706=90, srv1547802057=30, srv229070811=65, srv1840046500=40, srv2079422133=60, srv393258092=72, srv729410354=80, srv1917828270=46, srv1302589586=16, srv2143572639=64, srv367093400=69, srv2127410141=62} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-18 05:53:23,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-18 05:53:23,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-18 05:53:23,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-18 05:53:23,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-18 05:53:23,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-18 05:53:23,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-18 05:53:23,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-18 05:53:23,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-18 05:53:23,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-18 05:53:23,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-18 05:53:23,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-18 05:53:23,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-18 05:53:23,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-18 05:53:23,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-18 05:53:23,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-18 05:53:23,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-18 05:53:23,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-18 05:53:23,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-18 05:53:23,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-18 05:53:23,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-18 05:53:23,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-18 05:53:23,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-18 05:53:23,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-18 05:53:23,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-18 05:53:23,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-18 05:53:23,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-18 05:53:23,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-18 05:53:23,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-18 05:53:23,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-18 05:53:23,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-18 05:53:23,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-18 05:53:23,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-18 05:53:23,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-18 05:53:23,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-18 05:53:23,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-18 05:53:23,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-18 05:53:23,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-18 05:53:23,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-18 05:53:23,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-18 05:53:23,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-18 05:53:23,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-18 05:53:23,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-18 05:53:23,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-18 05:53:23,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-18 05:53:23,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-18 05:53:23,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-18 05:53:23,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-18 05:53:23,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-18 05:53:23,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-18 05:53:23,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-18 05:53:23,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-18 05:53:23,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-18 05:53:23,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-18 05:53:23,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-18 05:53:23,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-18 05:53:23,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-18 05:53:23,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-18 05:53:23,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-18 05:53:23,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-18 05:53:23,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-18 05:53:23,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-18 05:53:23,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-18 05:53:23,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-18 05:53:23,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-18 05:53:23,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-18 05:53:23,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-18 05:53:23,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-18 05:53:23,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-18 05:53:23,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-18 05:53:23,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-18 05:53:23,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-18 05:53:23,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-18 05:53:23,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-18 05:53:23,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-18 05:53:23,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-18 05:53:23,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-18 05:53:23,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-18 05:53:23,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-18 05:53:23,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-18 05:53:23,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-18 05:53:23,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-18 05:53:23,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-18 05:53:23,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-18 05:53:23,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-18 05:53:23,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-18 05:53:23,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-18 05:53:23,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-18 05:53:23,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-18 05:53:23,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-18 05:53:23,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-18 05:53:23,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-18 05:53:23,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-18 05:53:23,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-18 05:53:23,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-18 05:53:23,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-18 05:53:23,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-18 05:53:23,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-18 05:53:23,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-18 05:53:23,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-18 05:53:23,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-18 05:53:23,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-18 05:53:23,713 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-18 05:53:23,714 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-18 05:53:23,714 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-18 05:53:24,093 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 536 ms to try 5000 different iterations.  Found a solution that moves 4661 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18418104398860305. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04107666831530123, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.055353166702304064, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5178888888888888, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04107666831530123, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1795, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2455, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-18 05:53:24,113 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv729410354:81 , srv1316185371:82 , srv1391103828:82 , srv1547802057:82 , srv1059933114:83 , srv1126083564:83 , srv1205472562:83 , srv1917828270:83 , srv1948110338:83 , srv1968827491:83 , srv2079422133:83 , srv2128688614:83 , srv441955381:83 , srv740428352:83 , srv1065312256:84 , srv1171136028:84 , srv1403876356:84 , srv1414314498:84 , srv1607140967:84 , srv1665556060:84 , srv1769354690:84 , srv190520522:84 , srv1927644295:84 , srv1985270155:84 , srv2001536860:84 , srv2040979279:84 , srv1037359848:85 , srv1073304041:85 , srv108474473:85 , srv1188445380:85 , srv1211933749:85 , srv12200624:85 , srv1334277491:85 , srv1396081822:85 , srv1703543222:85 , srv1817548068:85 , srv1840046500:85 , srv33770308:85 , srv443972406:85 , srv875114548:85 , srv938368361:85 , srv944946571:85 , srv1082822503:86 , srv1302589586:86 , srv1643700617:86 , srv1798407439:86 , srv1844974711:86 , srv1865050520:86 , srv1998588546:86 , srv2114989359:86 , srv434508687:86 , srv563494468:86 , srv78668300:86 , srv840878023:86 , srv848748801:86 , srv861664772:86 , srv87545706:86 , srv917107262:86 , srv936577314:86 , srv990233025:86 , srv1068121823:87 , srv1419940182:87 , srv1439452758:87 , srv148886458:87 , srv1541837401:87 , srv1981090411:87 , srv1997529226:87 , srv2001704790:87 , srv2143572639:87 , srv812053759:87 , srv1125722661:88 , srv1332472545:88 , srv1862930635:88 , srv1945161511:88 , srv2127410141:88 , srv367093400:88 , srv48466051:88 , srv617226982:88 , srv722358233:88 , srv879047950:88 , srv1489115960:89 , srv1918749303:89 , srv229070811:89 , srv327058812:89 , srv791457266:89 , srv99945518:89 , srv1324369389:90 , srv1582282601:90 , srv187094629:90 , srv2075697194:90 , srv258804845:90 , srv369224354:90 , srv1097859539:91 , srv393258092:91 , srv376243884:92 , srv987441101:92 , srv913367377:93 , srv1837060085:94 , srv80432903:95 , srv1285867092:455 }
2023-08-18 05:53:24,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-18 05:53:24,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv148886458=27, srv1665556060=34, srv2114989359=61, srv1862930635=42, srv2040979279=58, srv1285867092=15, srv1865050520=43, srv2128688614=63, srv443972406=75, srv33770308=68, srv369224354=70, srv1419940182=25, srv1582282601=31, srv791457266=83, srv1188445380=11, srv48466051=76, srv1414314498=24, srv190520522=45, srv740428352=81, srv1059933114=1, srv12200624=14, srv840878023=86, srv376243884=71, srv1073304041=4, srv1334277491=20, srv1065312256=2, srv1205472562=12, srv1125722661=8, srv990233025=98, srv1037359848=0, srv1489115960=28, srv1211933749=13, srv258804845=66, srv936577314=94, srv848748801=87, srv2001536860=56, srv944946571=96, srv434508687=73, srv1981090411=52, srv1126083564=9, srv1998588546=55, srv1607140967=32, srv441955381=74, srv80432903=84, srv1324369389=18, srv1068121823=3, srv1837060085=39, srv2075697194=59, srv812053759=85, srv879047950=91, srv1927644295=48, srv1948110338=50, srv1097859539=7, srv108474473=6, srv875114548=89, srv1844974711=41, srv2001704790=57, srv917107262=93, srv563494468=77, srv1968827491=51, srv1997529226=54, srv1798407439=37, srv987441101=97, srv1403876356=23, srv617226982=78, srv1769354690=36, srv722358233=79, srv913367377=92, srv938368361=95, srv1643700617=33, srv1918749303=47, srv1082822503=5, srv1817548068=38, srv1396081822=22, srv1703543222=35, srv1332472545=19, srv861664772=88, srv327058812=67, srv1945161511=49, srv1985270155=53, srv1541837401=29, srv1171136028=10, srv1316185371=17, srv1391103828=21, srv1439452758=26, srv187094629=44, srv78668300=82, srv99945518=99, srv87545706=90, srv1547802057=30, srv229070811=65, srv1840046500=40, srv2079422133=60, srv393258092=72, srv729410354=80, srv1917828270=46, srv1302589586=16, srv2143572639=64, srv367093400=69, srv2127410141=62} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-18 05:53:24,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-18 05:53:24,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-18 05:53:24,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-18 05:53:24,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-18 05:53:24,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-18 05:53:24,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-18 05:53:24,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-18 05:53:24,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-18 05:53:24,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-18 05:53:24,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-18 05:53:24,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-18 05:53:24,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-18 05:53:24,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-18 05:53:24,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-18 05:53:24,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-18 05:53:24,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-18 05:53:24,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-18 05:53:24,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-18 05:53:24,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-18 05:53:24,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-18 05:53:24,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-18 05:53:24,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-18 05:53:24,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-18 05:53:24,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-18 05:53:24,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-18 05:53:24,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-18 05:53:24,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-18 05:53:24,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-18 05:53:24,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-18 05:53:24,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-18 05:53:24,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-18 05:53:24,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-18 05:53:24,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-18 05:53:24,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-18 05:53:24,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-18 05:53:24,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-18 05:53:24,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-18 05:53:24,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-18 05:53:24,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-18 05:53:24,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-18 05:53:24,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-18 05:53:24,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-18 05:53:24,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-18 05:53:24,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-18 05:53:24,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-18 05:53:24,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-18 05:53:24,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-18 05:53:24,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-18 05:53:24,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-18 05:53:24,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-18 05:53:24,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-18 05:53:24,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-18 05:53:24,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-18 05:53:24,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-18 05:53:24,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-18 05:53:24,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-18 05:53:24,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-18 05:53:24,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-18 05:53:24,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-18 05:53:24,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-18 05:53:24,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-18 05:53:24,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-18 05:53:24,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-18 05:53:24,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-18 05:53:24,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-18 05:53:24,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-18 05:53:24,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-18 05:53:24,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-18 05:53:24,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-18 05:53:24,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-18 05:53:24,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-18 05:53:24,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-18 05:53:24,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-18 05:53:24,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-18 05:53:24,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-18 05:53:24,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-18 05:53:24,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-18 05:53:24,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-18 05:53:24,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-18 05:53:24,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-18 05:53:24,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-18 05:53:24,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-18 05:53:24,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-18 05:53:24,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-18 05:53:24,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-18 05:53:24,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-18 05:53:24,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-18 05:53:24,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-18 05:53:24,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-18 05:53:24,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-18 05:53:24,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-18 05:53:24,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-18 05:53:24,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-18 05:53:24,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-18 05:53:24,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-18 05:53:24,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-18 05:53:24,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-18 05:53:24,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-18 05:53:24,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-18 05:53:24,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-18 05:53:24,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-18 05:53:24,322 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04107666831530123, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.055353166702304064, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04107666831530123, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1795, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2455, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-18 05:53:24,322 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-18 05:53:24,322 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18414840696903803, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04107666831530123, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.055353166702304064, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04107666831530123, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1795, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2455, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-18 05:53:24,693 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 518 ms to try 5000 different iterations.  Found a solution that moves 2696 regions; Going from a computed imbalance of 0.18414840696903803 to a new imbalance of 0.016573293132348215. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02173999792236758); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03436778696897403, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2995555555555556, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02173999792236758); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013416666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04683333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-18 05:53:24,708 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1837060085:78 , srv80432903:78 , srv376243884:80 , srv913367377:80 , srv987441101:80 , srv1125722661:81 , srv1324369389:81 , srv1918749303:81 , srv369224354:82 , srv393258092:82 , srv791457266:82 , srv99945518:82 , srv1097859539:83 , srv1332472545:83 , srv148886458:83 , srv1582282601:83 , srv1840046500:83 , srv1862930635:83 , srv1945161511:83 , srv2075697194:83 , srv229070811:83 , srv258804845:83 , srv327058812:83 , srv617226982:83 , srv1439452758:84 , srv1541837401:84 , srv1865050520:84 , srv187094629:84 , srv2001704790:84 , srv2127410141:84 , srv2143572639:84 , srv879047950:84 , srv990233025:84 , srv1037359848:85 , srv1082822503:85 , srv108474473:85 , srv1302589586:85 , srv1643700617:85 , srv1998588546:85 , srv367093400:85 , srv434508687:85 , srv443972406:85 , srv48466051:85 , srv78668300:85 , srv812053759:85 , srv848748801:85 , srv1065312256:86 , srv1068121823:86 , srv12200624:86 , srv1419940182:86 , srv1703543222:86 , srv1769354690:86 , srv1798407439:86 , srv1997529226:86 , srv2040979279:86 , srv2114989359:86 , srv563494468:86 , srv722358233:86 , srv87545706:86 , srv938368361:86 , srv1073304041:87 , srv1188445380:87 , srv1211933749:87 , srv1817548068:87 , srv1844974711:87 , srv190520522:87 , srv1981090411:87 , srv33770308:87 , srv861664772:87 , srv917107262:87 , srv936577314:87 , srv1205472562:88 , srv1391103828:88 , srv1403876356:88 , srv1607140967:88 , srv1927644295:88 , srv441955381:88 , srv840878023:88 , srv875114548:88 , srv944946571:88 , srv1059933114:89 , srv1334277491:89 , srv1396081822:89 , srv1414314498:89 , srv1917828270:89 , srv1948110338:89 , srv1985270155:89 , srv2001536860:89 , srv2079422133:89 , srv2128688614:89 , srv740428352:89 , srv1126083564:90 , srv1968827491:90 , srv1316185371:91 , srv1489115960:91 , srv1547802057:91 , srv1665556060:92 , srv729410354:92 , srv1171136028:95 , srv1285867092:512 }
2023-08-18 05:53:24,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-18 05:53:24,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv148886458=27, srv1665556060=34, srv2114989359=61, srv1862930635=42, srv2040979279=58, srv1285867092=15, srv1865050520=43, srv2128688614=63, srv443972406=75, srv33770308=68, srv369224354=70, srv1419940182=25, srv1582282601=31, srv791457266=83, srv1188445380=11, srv48466051=76, srv1414314498=24, srv190520522=45, srv740428352=81, srv1059933114=1, srv12200624=14, srv840878023=86, srv376243884=71, srv1073304041=4, srv1334277491=20, srv1065312256=2, srv1205472562=12, srv1125722661=8, srv990233025=98, srv1037359848=0, srv1489115960=28, srv1211933749=13, srv258804845=66, srv936577314=94, srv848748801=87, srv2001536860=56, srv944946571=96, srv434508687=73, srv1981090411=52, srv1126083564=9, srv1998588546=55, srv1607140967=32, srv441955381=74, srv80432903=84, srv1324369389=18, srv1068121823=3, srv1837060085=39, srv2075697194=59, srv812053759=85, srv879047950=91, srv1927644295=48, srv1948110338=50, srv1097859539=7, srv108474473=6, srv875114548=89, srv1844974711=41, srv2001704790=57, srv917107262=93, srv563494468=77, srv1968827491=51, srv1997529226=54, srv1798407439=37, srv987441101=97, srv1403876356=23, srv617226982=78, srv1769354690=36, srv722358233=79, srv913367377=92, srv938368361=95, srv1643700617=33, srv1918749303=47, srv1082822503=5, srv1817548068=38, srv1396081822=22, srv1703543222=35, srv1332472545=19, srv861664772=88, srv327058812=67, srv1945161511=49, srv1985270155=53, srv1541837401=29, srv1171136028=10, srv1316185371=17, srv1391103828=21, srv1439452758=26, srv187094629=44, srv78668300=82, srv99945518=99, srv87545706=90, srv1547802057=30, srv229070811=65, srv1840046500=40, srv2079422133=60, srv393258092=72, srv729410354=80, srv1917828270=46, srv1302589586=16, srv2143572639=64, srv367093400=69, srv2127410141=62} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-18 05:53:24,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-18 05:53:24,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-18 05:53:24,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-18 05:53:24,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-18 05:53:24,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-18 05:53:24,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-18 05:53:24,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-18 05:53:24,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-18 05:53:24,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-18 05:53:24,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-18 05:53:24,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-18 05:53:24,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-18 05:53:24,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-18 05:53:24,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-18 05:53:24,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-18 05:53:24,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-18 05:53:24,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-18 05:53:24,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-18 05:53:24,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-18 05:53:24,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-18 05:53:24,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-18 05:53:24,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-18 05:53:24,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-18 05:53:24,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-18 05:53:24,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-18 05:53:24,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-18 05:53:24,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-18 05:53:24,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-18 05:53:24,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-18 05:53:24,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-18 05:53:24,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-18 05:53:24,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-18 05:53:24,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-18 05:53:24,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-18 05:53:24,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-18 05:53:24,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-18 05:53:24,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-18 05:53:24,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-18 05:53:24,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-18 05:53:24,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-18 05:53:24,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-18 05:53:24,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-18 05:53:24,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-18 05:53:24,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-18 05:53:24,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-18 05:53:24,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-18 05:53:24,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-18 05:53:24,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-18 05:53:24,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-18 05:53:24,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-18 05:53:24,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-18 05:53:24,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-18 05:53:24,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-18 05:53:24,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-18 05:53:24,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-18 05:53:24,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-18 05:53:24,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-18 05:53:24,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-18 05:53:24,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-18 05:53:24,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-18 05:53:24,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-18 05:53:24,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-18 05:53:24,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-18 05:53:24,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-18 05:53:24,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-18 05:53:24,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-18 05:53:24,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-18 05:53:24,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-18 05:53:24,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-18 05:53:24,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-18 05:53:24,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-18 05:53:24,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-18 05:53:24,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-18 05:53:24,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-18 05:53:24,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-18 05:53:24,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-18 05:53:24,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-18 05:53:24,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-18 05:53:24,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-18 05:53:24,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-18 05:53:24,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-18 05:53:24,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-18 05:53:24,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-18 05:53:24,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-18 05:53:24,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-18 05:53:24,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-18 05:53:24,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-18 05:53:24,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-18 05:53:24,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-18 05:53:24,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-18 05:53:24,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-18 05:53:24,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-18 05:53:24,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-18 05:53:24,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-18 05:53:24,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-18 05:53:24,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-18 05:53:24,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-18 05:53:24,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-18 05:53:24,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-18 05:53:24,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-18 05:53:24,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-18 05:53:24,905 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02173999792236758); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03436778696897403, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02173999792236758); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013416666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04683333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-18 05:53:24,905 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-18 05:53:24,906 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.016554415336865, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02173999792236758); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03436778696897403, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02173999792236758); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013416666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04683333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-18 05:53:25,193 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 402 ms to try 5000 different iterations.  Found a solution that moves 948 regions; Going from a computed imbalance of 0.016554415336865 to a new imbalance of 8.278212222725098E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00879580895336142); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.011350946899297087); MoveCostFunction : (multiplier=7.0, imbalance=0.10533333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.00879580895336142); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00475); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-18 05:53:25,195 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1171136028:77 , srv1489115960:77 , srv1665556060:82 , srv1097859539:83 , srv1396081822:83 , srv1547802057:83 , srv1981090411:83 , srv1985270155:83 , srv1997529226:83 , srv840878023:83 , srv1059933114:84 , srv1068121823:84 , srv1188445380:84 , srv1316185371:84 , srv1414314498:84 , srv1419940182:84 , srv1582282601:84 , srv1844974711:84 , srv187094629:84 , srv1968827491:84 , srv2001536860:84 , srv2075697194:84 , srv2128688614:84 , srv327058812:84 , srv393258092:84 , srv48466051:84 , srv563494468:84 , srv722358233:84 , srv729410354:84 , srv740428352:84 , srv80432903:84 , srv861664772:84 , srv913367377:84 , srv936577314:84 , srv1073304041:85 , srv1126083564:85 , srv1211933749:85 , srv1334277491:85 , srv1403876356:85 , srv1541837401:85 , srv1798407439:85 , srv1837060085:85 , srv1917828270:85 , srv1948110338:85 , srv1998588546:85 , srv2114989359:85 , srv229070811:85 , srv367093400:85 , srv369224354:85 , srv434508687:85 , srv812053759:85 , srv875114548:85 , srv879047950:85 , srv944946571:85 , srv1082822503:86 , srv1205472562:86 , srv12200624:86 , srv1302589586:86 , srv1324369389:86 , srv1332472545:86 , srv1643700617:86 , srv1703543222:86 , srv190520522:86 , srv1927644295:86 , srv2001704790:86 , srv2079422133:86 , srv2127410141:86 , srv258804845:86 , srv33770308:86 , srv441955381:86 , srv617226982:86 , srv791457266:86 , srv848748801:86 , srv87545706:86 , srv917107262:86 , srv938368361:86 , srv99945518:86 , srv1037359848:87 , srv1065312256:87 , srv108474473:87 , srv1391103828:87 , srv1439452758:87 , srv148886458:87 , srv1607140967:87 , srv1769354690:87 , srv1817548068:87 , srv1862930635:87 , srv1865050520:87 , srv1918749303:87 , srv1945161511:87 , srv376243884:87 , srv443972406:87 , srv78668300:87 , srv987441101:87 , srv990233025:87 , srv1125722661:88 , srv2040979279:88 , srv2143572639:88 , srv1840046500:90 , srv1285867092:569 }
2023-08-18 05:53:25,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-18 05:53:25,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv148886458=27, srv1665556060=34, srv2114989359=61, srv1862930635=42, srv2040979279=58, srv1285867092=15, srv1865050520=43, srv2128688614=63, srv443972406=75, srv33770308=68, srv369224354=70, srv1419940182=25, srv1582282601=31, srv791457266=83, srv1188445380=11, srv48466051=76, srv1414314498=24, srv190520522=45, srv740428352=81, srv1059933114=1, srv12200624=14, srv840878023=86, srv376243884=71, srv1073304041=4, srv1334277491=20, srv1065312256=2, srv1205472562=12, srv1125722661=8, srv990233025=98, srv1037359848=0, srv1489115960=28, srv1211933749=13, srv258804845=66, srv936577314=94, srv848748801=87, srv2001536860=56, srv944946571=96, srv434508687=73, srv1981090411=52, srv1126083564=9, srv1998588546=55, srv1607140967=32, srv441955381=74, srv80432903=84, srv1324369389=18, srv1068121823=3, srv1837060085=39, srv2075697194=59, srv812053759=85, srv879047950=91, srv1927644295=48, srv1948110338=50, srv1097859539=7, srv108474473=6, srv875114548=89, srv1844974711=41, srv2001704790=57, srv917107262=93, srv563494468=77, srv1968827491=51, srv1997529226=54, srv1798407439=37, srv987441101=97, srv1403876356=23, srv617226982=78, srv1769354690=36, srv722358233=79, srv913367377=92, srv938368361=95, srv1643700617=33, srv1918749303=47, srv1082822503=5, srv1817548068=38, srv1396081822=22, srv1703543222=35, srv1332472545=19, srv861664772=88, srv327058812=67, srv1945161511=49, srv1985270155=53, srv1541837401=29, srv1171136028=10, srv1316185371=17, srv1391103828=21, srv1439452758=26, srv187094629=44, srv78668300=82, srv99945518=99, srv87545706=90, srv1547802057=30, srv229070811=65, srv1840046500=40, srv2079422133=60, srv393258092=72, srv729410354=80, srv1917828270=46, srv1302589586=16, srv2143572639=64, srv367093400=69, srv2127410141=62} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-18 05:53:25,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-18 05:53:25,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-18 05:53:25,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-18 05:53:25,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-18 05:53:25,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-18 05:53:25,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-18 05:53:25,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-18 05:53:25,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-18 05:53:25,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-18 05:53:25,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-18 05:53:25,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-18 05:53:25,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-18 05:53:25,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-18 05:53:25,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-18 05:53:25,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-18 05:53:25,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-18 05:53:25,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-18 05:53:25,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-18 05:53:25,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-18 05:53:25,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-18 05:53:25,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-18 05:53:25,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-18 05:53:25,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-18 05:53:25,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-18 05:53:25,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-18 05:53:25,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-18 05:53:25,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-18 05:53:25,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-18 05:53:25,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-18 05:53:25,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-18 05:53:25,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-18 05:53:25,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-18 05:53:25,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-18 05:53:25,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-18 05:53:25,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-18 05:53:25,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-18 05:53:25,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-18 05:53:25,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-18 05:53:25,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-18 05:53:25,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-18 05:53:25,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-18 05:53:25,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-18 05:53:25,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-18 05:53:25,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-18 05:53:25,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-18 05:53:25,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-18 05:53:25,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-18 05:53:25,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-18 05:53:25,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-18 05:53:25,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-18 05:53:25,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-18 05:53:25,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-18 05:53:25,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-18 05:53:25,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-18 05:53:25,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-18 05:53:25,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-18 05:53:25,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-18 05:53:25,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-18 05:53:25,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-18 05:53:25,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-18 05:53:25,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-18 05:53:25,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-18 05:53:25,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-18 05:53:25,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-18 05:53:25,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-18 05:53:25,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-18 05:53:25,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-18 05:53:25,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-18 05:53:25,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-18 05:53:25,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-18 05:53:25,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-18 05:53:25,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-18 05:53:25,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-18 05:53:25,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-18 05:53:25,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-18 05:53:25,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-18 05:53:25,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-18 05:53:25,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-18 05:53:25,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-18 05:53:25,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-18 05:53:25,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-18 05:53:25,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-18 05:53:25,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-18 05:53:25,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-18 05:53:25,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-18 05:53:25,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-18 05:53:25,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-18 05:53:25,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-18 05:53:25,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-18 05:53:25,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-18 05:53:25,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-18 05:53:25,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-18 05:53:25,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-18 05:53:25,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-18 05:53:25,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-18 05:53:25,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-18 05:53:25,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-18 05:53:25,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-18 05:53:25,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-18 05:53:25,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-18 05:53:25,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-18 05:53:25,376 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00879580895336142); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.011350946899297087); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.00879580895336142); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00475); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-18 05:53:25,376 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-18 05:53:25,376 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=8.211831843948813E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00879580895336142); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.011350946899297087); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.00879580895336142); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00475); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-18 05:53:25,849 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 616 ms to try 5000 different iterations.  Found a solution that moves 273 regions; Going from a computed imbalance of 8.211831843948813E-4 to a new imbalance of 4.298140696278718E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.178341294946416E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.030333333333333334, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.178341294946416E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-18 05:53:25,850 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1334277491:83 , srv1607140967:83 , srv1817548068:83 , srv2143572639:83 , srv258804845:83 , srv376243884:83 , srv917107262:83 , srv987441101:83 , srv1126083564:84 , srv1316185371:84 , srv1439452758:84 , srv1665556060:84 , srv1840046500:84 , srv1862930635:84 , srv187094629:84 , srv1918749303:84 , srv1927644295:84 , srv1945161511:84 , srv2040979279:84 , srv2127410141:84 , srv33770308:84 , srv367093400:84 , srv722358233:84 , srv78668300:84 , srv791457266:84 , srv875114548:84 , srv87545706:84 , srv944946571:84 , srv1037359848:85 , srv1059933114:85 , srv1065312256:85 , srv1068121823:85 , srv1073304041:85 , srv1082822503:85 , srv108474473:85 , srv1097859539:85 , srv1125722661:85 , srv1188445380:85 , srv1205472562:85 , srv1211933749:85 , srv12200624:85 , srv1324369389:85 , srv1332472545:85 , srv1391103828:85 , srv1396081822:85 , srv1403876356:85 , srv1414314498:85 , srv1419940182:85 , srv148886458:85 , srv1582282601:85 , srv1643700617:85 , srv1703543222:85 , srv1798407439:85 , srv1837060085:85 , srv1844974711:85 , srv1865050520:85 , srv190520522:85 , srv1917828270:85 , srv1948110338:85 , srv1968827491:85 , srv1981090411:85 , srv1997529226:85 , srv1998588546:85 , srv2001536860:85 , srv2001704790:85 , srv2075697194:85 , srv2079422133:85 , srv2114989359:85 , srv229070811:85 , srv369224354:85 , srv393258092:85 , srv441955381:85 , srv443972406:85 , srv48466051:85 , srv729410354:85 , srv812053759:85 , srv840878023:85 , srv848748801:85 , srv861664772:85 , srv879047950:85 , srv913367377:85 , srv938368361:85 , srv990233025:85 , srv99945518:85 , srv1171136028:86 , srv1302589586:86 , srv1489115960:86 , srv1541837401:86 , srv1547802057:86 , srv1769354690:86 , srv1985270155:86 , srv2128688614:86 , srv327058812:86 , srv434508687:86 , srv563494468:86 , srv617226982:86 , srv740428352:86 , srv80432903:86 , srv936577314:86 , srv1285867092:606 }
2023-08-18 05:53:25,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-18 05:53:25,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv148886458=27, srv1665556060=34, srv2114989359=61, srv1862930635=42, srv2040979279=58, srv1285867092=15, srv1865050520=43, srv2128688614=63, srv443972406=75, srv33770308=68, srv369224354=70, srv1419940182=25, srv1582282601=31, srv791457266=83, srv1188445380=11, srv48466051=76, srv1414314498=24, srv190520522=45, srv740428352=81, srv1059933114=1, srv12200624=14, srv840878023=86, srv376243884=71, srv1073304041=4, srv1334277491=20, srv1065312256=2, srv1205472562=12, srv1125722661=8, srv990233025=98, srv1037359848=0, srv1489115960=28, srv1211933749=13, srv258804845=66, srv936577314=94, srv848748801=87, srv2001536860=56, srv944946571=96, srv434508687=73, srv1981090411=52, srv1126083564=9, srv1998588546=55, srv1607140967=32, srv441955381=74, srv80432903=84, srv1324369389=18, srv1068121823=3, srv1837060085=39, srv2075697194=59, srv812053759=85, srv879047950=91, srv1927644295=48, srv1948110338=50, srv1097859539=7, srv108474473=6, srv875114548=89, srv1844974711=41, srv2001704790=57, srv917107262=93, srv563494468=77, srv1968827491=51, srv1997529226=54, srv1798407439=37, srv987441101=97, srv1403876356=23, srv617226982=78, srv1769354690=36, srv722358233=79, srv913367377=92, srv938368361=95, srv1643700617=33, srv1918749303=47, srv1082822503=5, srv1817548068=38, srv1396081822=22, srv1703543222=35, srv1332472545=19, srv861664772=88, srv327058812=67, srv1945161511=49, srv1985270155=53, srv1541837401=29, srv1171136028=10, srv1316185371=17, srv1391103828=21, srv1439452758=26, srv187094629=44, srv78668300=82, srv99945518=99, srv87545706=90, srv1547802057=30, srv229070811=65, srv1840046500=40, srv2079422133=60, srv393258092=72, srv729410354=80, srv1917828270=46, srv1302589586=16, srv2143572639=64, srv367093400=69, srv2127410141=62} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-18 05:53:25,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-18 05:53:25,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-18 05:53:25,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-18 05:53:25,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-18 05:53:25,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-18 05:53:25,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-18 05:53:25,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-18 05:53:25,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-18 05:53:25,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-18 05:53:25,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-18 05:53:25,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-18 05:53:25,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-18 05:53:25,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-18 05:53:25,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-18 05:53:25,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-18 05:53:25,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-18 05:53:25,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-18 05:53:25,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-18 05:53:25,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-18 05:53:25,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-18 05:53:25,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-18 05:53:25,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-18 05:53:25,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-18 05:53:25,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-18 05:53:25,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-18 05:53:25,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-18 05:53:25,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-18 05:53:25,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-18 05:53:25,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-18 05:53:25,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-18 05:53:25,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-18 05:53:25,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-18 05:53:25,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-18 05:53:25,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-18 05:53:25,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-18 05:53:25,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-18 05:53:25,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-18 05:53:25,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-18 05:53:25,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-18 05:53:25,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-18 05:53:25,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-18 05:53:25,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-18 05:53:25,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-18 05:53:25,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-18 05:53:25,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-18 05:53:25,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-18 05:53:25,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-18 05:53:25,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-18 05:53:25,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-18 05:53:25,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-18 05:53:25,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-18 05:53:25,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-18 05:53:25,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-18 05:53:25,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-18 05:53:25,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-18 05:53:25,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-18 05:53:25,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-18 05:53:25,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-18 05:53:25,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-18 05:53:25,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-18 05:53:25,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-18 05:53:25,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-18 05:53:25,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-18 05:53:25,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-18 05:53:25,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-18 05:53:25,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-18 05:53:25,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-18 05:53:25,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-18 05:53:25,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-18 05:53:25,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-18 05:53:25,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-18 05:53:25,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-18 05:53:25,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-18 05:53:25,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-18 05:53:25,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-18 05:53:25,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-18 05:53:25,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-18 05:53:25,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-18 05:53:25,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-18 05:53:25,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-18 05:53:25,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-18 05:53:25,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-18 05:53:25,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-18 05:53:25,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-18 05:53:25,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-18 05:53:25,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-18 05:53:25,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-18 05:53:25,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-18 05:53:25,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-18 05:53:25,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-18 05:53:25,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-18 05:53:25,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-18 05:53:25,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-18 05:53:25,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-18 05:53:25,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-18 05:53:25,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-18 05:53:25,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-18 05:53:25,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-18 05:53:25,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-18 05:53:25,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-18 05:53:25,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-18 05:53:26,011 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.178341294946416E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.178341294946416E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-18 05:53:26,012 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-18 05:53:26,012 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=4.106982010562203E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.178341294946416E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.178341294946416E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-18 05:53:26,268 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 377 ms to try 5000 different iterations.  Found a solution that moves 27 regions; Going from a computed imbalance of 4.106982010562203E-5 to a new imbalance of 7.57681554113109E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0016412198797244363); MoveCostFunction : (multiplier=7.0, imbalance=0.003); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-18 05:53:26,269 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1302589586:83 , srv1489115960:83 , srv1769354690:83 , srv2079422133:83 , srv617226982:83 , srv80432903:83 , srv936577314:83 , srv1037359848:84 , srv1065312256:84 , srv1068121823:84 , srv1073304041:84 , srv1082822503:84 , srv108474473:84 , srv1097859539:84 , srv1125722661:84 , srv1126083564:84 , srv1171136028:84 , srv1205472562:84 , srv1211933749:84 , srv12200624:84 , srv1324369389:84 , srv1332472545:84 , srv1334277491:84 , srv1391103828:84 , srv1396081822:84 , srv1403876356:84 , srv1414314498:84 , srv1419940182:84 , srv1439452758:84 , srv148886458:84 , srv1541837401:84 , srv1547802057:84 , srv1582282601:84 , srv1607140967:84 , srv1643700617:84 , srv1665556060:84 , srv1703543222:84 , srv1798407439:84 , srv1817548068:84 , srv1837060085:84 , srv1840046500:84 , srv1844974711:84 , srv1862930635:84 , srv1865050520:84 , srv187094629:84 , srv190520522:84 , srv1917828270:84 , srv1927644295:84 , srv1945161511:84 , srv1948110338:84 , srv1968827491:84 , srv1981090411:84 , srv1985270155:84 , srv2001536860:84 , srv2001704790:84 , srv2040979279:84 , srv2075697194:84 , srv2114989359:84 , srv2127410141:84 , srv2128688614:84 , srv2143572639:84 , srv229070811:84 , srv258804845:84 , srv327058812:84 , srv33770308:84 , srv367093400:84 , srv369224354:84 , srv376243884:84 , srv393258092:84 , srv434508687:84 , srv441955381:84 , srv443972406:84 , srv48466051:84 , srv563494468:84 , srv722358233:84 , srv729410354:84 , srv740428352:84 , srv78668300:84 , srv812053759:84 , srv840878023:84 , srv848748801:84 , srv861664772:84 , srv875114548:84 , srv87545706:84 , srv879047950:84 , srv913367377:84 , srv917107262:84 , srv938368361:84 , srv944946571:84 , srv987441101:84 , srv990233025:84 , srv99945518:84 , srv1059933114:85 , srv1188445380:85 , srv1316185371:85 , srv1918749303:85 , srv1997529226:85 , srv1998588546:85 , srv791457266:85 , srv1285867092:684 }
2023-08-18 05:53:26,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-18 05:53:26,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv148886458=27, srv1665556060=34, srv2114989359=61, srv1862930635=42, srv2040979279=58, srv1285867092=15, srv1865050520=43, srv2128688614=63, srv443972406=75, srv33770308=68, srv369224354=70, srv1419940182=25, srv1582282601=31, srv791457266=83, srv1188445380=11, srv48466051=76, srv1414314498=24, srv190520522=45, srv740428352=81, srv1059933114=1, srv12200624=14, srv840878023=86, srv376243884=71, srv1073304041=4, srv1334277491=20, srv1065312256=2, srv1205472562=12, srv1125722661=8, srv990233025=98, srv1037359848=0, srv1489115960=28, srv1211933749=13, srv258804845=66, srv936577314=94, srv848748801=87, srv2001536860=56, srv944946571=96, srv434508687=73, srv1981090411=52, srv1126083564=9, srv1998588546=55, srv1607140967=32, srv441955381=74, srv80432903=84, srv1324369389=18, srv1068121823=3, srv1837060085=39, srv2075697194=59, srv812053759=85, srv879047950=91, srv1927644295=48, srv1948110338=50, srv1097859539=7, srv108474473=6, srv875114548=89, srv1844974711=41, srv2001704790=57, srv917107262=93, srv563494468=77, srv1968827491=51, srv1997529226=54, srv1798407439=37, srv987441101=97, srv1403876356=23, srv617226982=78, srv1769354690=36, srv722358233=79, srv913367377=92, srv938368361=95, srv1643700617=33, srv1918749303=47, srv1082822503=5, srv1817548068=38, srv1396081822=22, srv1703543222=35, srv1332472545=19, srv861664772=88, srv327058812=67, srv1945161511=49, srv1985270155=53, srv1541837401=29, srv1171136028=10, srv1316185371=17, srv1391103828=21, srv1439452758=26, srv187094629=44, srv78668300=82, srv99945518=99, srv87545706=90, srv1547802057=30, srv229070811=65, srv1840046500=40, srv2079422133=60, srv393258092=72, srv729410354=80, srv1917828270=46, srv1302589586=16, srv2143572639=64, srv367093400=69, srv2127410141=62} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-18 05:53:26,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-18 05:53:26,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-18 05:53:26,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-18 05:53:26,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-18 05:53:26,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-18 05:53:26,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-18 05:53:26,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-18 05:53:26,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-18 05:53:26,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-18 05:53:26,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-18 05:53:26,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-18 05:53:26,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-18 05:53:26,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-18 05:53:26,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-18 05:53:26,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-18 05:53:26,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-18 05:53:26,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-18 05:53:26,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-18 05:53:26,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-18 05:53:26,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-18 05:53:26,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-18 05:53:26,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-18 05:53:26,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-18 05:53:26,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-18 05:53:26,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-18 05:53:26,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-18 05:53:26,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-18 05:53:26,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-18 05:53:26,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-18 05:53:26,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-18 05:53:26,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-18 05:53:26,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-18 05:53:26,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-18 05:53:26,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-18 05:53:26,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-18 05:53:26,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-18 05:53:26,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-18 05:53:26,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-18 05:53:26,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-18 05:53:26,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-18 05:53:26,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-18 05:53:26,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-18 05:53:26,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-18 05:53:26,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-18 05:53:26,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-18 05:53:26,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-18 05:53:26,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-18 05:53:26,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-18 05:53:26,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-18 05:53:26,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-18 05:53:26,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-18 05:53:26,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-18 05:53:26,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-18 05:53:26,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-18 05:53:26,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-18 05:53:26,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-18 05:53:26,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-18 05:53:26,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-18 05:53:26,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-18 05:53:26,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-18 05:53:26,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-18 05:53:26,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-18 05:53:26,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-18 05:53:26,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-18 05:53:26,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-18 05:53:26,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-18 05:53:26,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-18 05:53:26,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-18 05:53:26,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-18 05:53:26,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-18 05:53:26,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-18 05:53:26,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-18 05:53:26,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-18 05:53:26,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-18 05:53:26,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-18 05:53:26,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-18 05:53:26,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-18 05:53:26,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-18 05:53:26,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-18 05:53:26,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-18 05:53:26,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-18 05:53:26,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-18 05:53:26,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-18 05:53:26,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-18 05:53:26,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-18 05:53:26,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-18 05:53:26,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-18 05:53:26,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-18 05:53:26,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-18 05:53:26,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-18 05:53:26,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-18 05:53:26,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-18 05:53:26,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-18 05:53:26,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-18 05:53:26,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-18 05:53:26,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-18 05:53:26,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-18 05:53:26,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-18 05:53:26,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-18 05:53:26,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-18 05:53:26,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-18 05:53:26,451 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=7.387757500312559E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0016412198797244363); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-18 05:53:26,451 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1302589586:83 , srv1489115960:83 , srv1769354690:83 , srv2079422133:83 , srv617226982:83 , srv80432903:83 , srv936577314:83 , srv1037359848:84 , srv1065312256:84 , srv1068121823:84 , srv1073304041:84 , srv1082822503:84 , srv108474473:84 , srv1097859539:84 , srv1125722661:84 , srv1126083564:84 , srv1171136028:84 , srv1205472562:84 , srv1211933749:84 , srv12200624:84 , srv1324369389:84 , srv1332472545:84 , srv1334277491:84 , srv1391103828:84 , srv1396081822:84 , srv1403876356:84 , srv1414314498:84 , srv1419940182:84 , srv1439452758:84 , srv148886458:84 , srv1541837401:84 , srv1547802057:84 , srv1582282601:84 , srv1607140967:84 , srv1643700617:84 , srv1665556060:84 , srv1703543222:84 , srv1798407439:84 , srv1817548068:84 , srv1837060085:84 , srv1840046500:84 , srv1844974711:84 , srv1862930635:84 , srv1865050520:84 , srv187094629:84 , srv190520522:84 , srv1917828270:84 , srv1927644295:84 , srv1945161511:84 , srv1948110338:84 , srv1968827491:84 , srv1981090411:84 , srv1985270155:84 , srv2001536860:84 , srv2001704790:84 , srv2040979279:84 , srv2075697194:84 , srv2114989359:84 , srv2127410141:84 , srv2128688614:84 , srv2143572639:84 , srv229070811:84 , srv258804845:84 , srv327058812:84 , srv33770308:84 , srv367093400:84 , srv369224354:84 , srv376243884:84 , srv393258092:84 , srv434508687:84 , srv441955381:84 , srv443972406:84 , srv48466051:84 , srv563494468:84 , srv722358233:84 , srv729410354:84 , srv740428352:84 , srv78668300:84 , srv812053759:84 , srv840878023:84 , srv848748801:84 , srv861664772:84 , srv875114548:84 , srv87545706:84 , srv879047950:84 , srv913367377:84 , srv917107262:84 , srv938368361:84 , srv944946571:84 , srv987441101:84 , srv990233025:84 , srv99945518:84 , srv1059933114:85 , srv1188445380:85 , srv1316185371:85 , srv1918749303:85 , srv1997529226:85 , srv1998588546:85 , srv791457266:85 , srv1285867092:684 }
2023-08-18 05:53:26,480 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=242 (was 220) - SystemLoadAverage LEAK? -, ProcessCount=176 (was 176), AvailableMemoryMB=2914 (was 3768)
2023-08-18 05:53:26,494 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=242, ProcessCount=176, AvailableMemoryMB=2913
2023-08-18 05:53:26,495 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-18 05:53:26,495 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-08-18 05:53:26,508 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1014746361:84 , srv1019302661:84 , srv1047101180:84 , srv1062588134:84 , srv1095986180:84 , srv1142367389:84 , srv1179691528:84 , srv1368298443:84 , srv1393042923:84 , srv1526981989:84 , srv15666701:84 , srv1621706043:84 , srv1754036103:84 , srv1755748322:84 , srv1907653358:84 , srv1926859844:84 , srv2133634955:84 , srv218570166:84 , srv249184767:84 , srv28377963:84 , srv452732314:84 , srv510923638:84 , srv6046364:84 , srv605032522:84 , srv655562151:84 , srv688400045:84 , srv725652514:84 , srv852412359:84 , srv901079237:84 , srv454453399:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-08-18 05:53:26,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-18 05:53:26,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1095986180=4, srv1047101180=2, srv655562151=25, srv218570166=17, srv452732314=20, srv725652514=27, srv1142367389=5, srv1393042923=8, srv1062588134=3, srv510923638=22, srv688400045=26, srv2133634955=16, srv1368298443=7, srv249184767=18, srv901079237=29, srv1621706043=11, srv15666701=10, srv1754036103=12, srv1179691528=6, srv1755748322=13, srv1907653358=14, srv6046364=23, srv1526981989=9, srv1926859844=15, srv852412359=28, srv605032522=24, srv28377963=19, srv454453399=21, srv1014746361=0, srv1019302661=1} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-18 05:53:26,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-18 05:53:26,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-18 05:53:26,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-18 05:53:26,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-18 05:53:26,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-18 05:53:26,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-18 05:53:26,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-18 05:53:26,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-18 05:53:26,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-18 05:53:26,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-18 05:53:26,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-18 05:53:26,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-18 05:53:26,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-18 05:53:26,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-18 05:53:26,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-18 05:53:26,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-18 05:53:26,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-18 05:53:26,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-18 05:53:26,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-18 05:53:26,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-18 05:53:26,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-18 05:53:26,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-18 05:53:26,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-18 05:53:26,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-18 05:53:26,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-18 05:53:26,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-18 05:53:26,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-18 05:53:26,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-18 05:53:26,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-18 05:53:26,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-18 05:53:26,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-08-18 05:53:26,552 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1001149180848309, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-18 05:53:26,553 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912508502102113, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1001149180848309, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-08-18 05:55:26,530 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5887718 different iterations.  Found a solution that moves 1875 regions; Going from a computed imbalance of 0.9912508502102113 to a new imbalance of 4.376343537466002E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6944444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-18 05:55:26,538 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1014746361:90 , srv1019302661:90 , srv1047101180:90 , srv1062588134:90 , srv1095986180:90 , srv1142367389:90 , srv1179691528:90 , srv1368298443:90 , srv1393042923:90 , srv1526981989:90 , srv15666701:90 , srv1621706043:90 , srv1754036103:90 , srv1755748322:90 , srv1907653358:90 , srv1926859844:90 , srv2133634955:90 , srv218570166:90 , srv249184767:90 , srv28377963:90 , srv452732314:90 , srv454453399:90 , srv510923638:90 , srv6046364:90 , srv605032522:90 , srv655562151:90 , srv688400045:90 , srv725652514:90 , srv852412359:90 , srv901079237:90 }
2023-08-18 05:55:26,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-18 05:55:26,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1095986180=4, srv1047101180=2, srv655562151=25, srv218570166=17, srv452732314=20, srv725652514=27, srv1142367389=5, srv1393042923=8, srv1062588134=3, srv510923638=22, srv688400045=26, srv2133634955=16, srv1368298443=7, srv249184767=18, srv901079237=29, srv1621706043=11, srv15666701=10, srv1754036103=12, srv1179691528=6, srv1755748322=13, srv1907653358=14, srv6046364=23, srv1526981989=9, srv1926859844=15, srv852412359=28, srv605032522=24, srv28377963=19, srv454453399=21, srv1014746361=0, srv1019302661=1} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-18 05:55:26,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-18 05:55:26,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-18 05:55:26,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-18 05:55:26,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-18 05:55:26,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-18 05:55:26,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-18 05:55:26,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-18 05:55:26,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-18 05:55:26,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-18 05:55:26,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-18 05:55:26,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-18 05:55:26,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-18 05:55:26,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-18 05:55:26,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-18 05:55:26,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-18 05:55:26,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-18 05:55:26,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-18 05:55:26,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-18 05:55:26,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-18 05:55:26,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-18 05:55:26,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-18 05:55:26,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-18 05:55:26,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-18 05:55:26,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-18 05:55:26,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-18 05:55:26,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-18 05:55:26,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-18 05:55:26,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-18 05:55:26,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-18 05:55:26,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-18 05:55:26,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-08-18 05:55:26,563 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-18 05:55:26,563 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1014746361:90 , srv1019302661:90 , srv1047101180:90 , srv1062588134:90 , srv1095986180:90 , srv1142367389:90 , srv1179691528:90 , srv1368298443:90 , srv1393042923:90 , srv1526981989:90 , srv15666701:90 , srv1621706043:90 , srv1754036103:90 , srv1755748322:90 , srv1907653358:90 , srv1926859844:90 , srv2133634955:90 , srv218570166:90 , srv249184767:90 , srv28377963:90 , srv452732314:90 , srv454453399:90 , srv510923638:90 , srv6046364:90 , srv605032522:90 , srv655562151:90 , srv688400045:90 , srv725652514:90 , srv852412359:90 , srv901079237:90 }
2023-08-18 05:55:26,580 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=189 (was 242), ProcessCount=173 (was 176), AvailableMemoryMB=2792 (was 2913)