2023-08-23 02:53:28,687 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-08-23 02:53:29,024 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-08-23 02:53:29,138 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-08-23 02:53:29,205 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-23 02:53:29,213 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-08-23 02:53:29,228 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=294, ProcessCount=171, AvailableMemoryMB=12367
2023-08-23 02:53:29,230 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-23 02:53:29,231 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-08-23 02:53:29,376 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1018822835:84 , srv1055429117:84 , srv1067545616:84 , srv1071460766:84 , srv1075283504:84 , srv1087380909:84 , srv1094119809:84 , srv1094863227:84 , srv1099637395:84 , srv1101025502:84 , srv1121782692:84 , srv1122734407:84 , srv1130795841:84 , srv115228089:84 , srv1167213891:84 , srv1194436131:84 , srv1196117367:84 , srv1213520811:84 , srv1265970916:84 , srv1390577757:84 , srv1392262761:84 , srv1401805386:84 , srv1424312998:84 , srv1441725338:84 , srv1449102055:84 , srv1454838615:84 , srv1478477389:84 , srv1515187950:84 , srv1516630381:84 , srv1524216183:84 , srv1524859000:84 , srv1535088183:84 , srv1578064349:84 , srv1613607997:84 , srv1637079397:84 , srv1642254832:84 , srv1643681157:84 , srv1671302576:84 , srv1675406261:84 , srv1682881721:84 , srv1691062550:84 , srv1697481338:84 , srv1715588858:84 , srv1763378078:84 , srv1798708615:84 , srv1855940610:84 , srv1864728118:84 , srv1902184167:84 , srv1919674422:84 , srv1942054986:84 , srv1970965449:84 , srv2014128489:84 , srv20363050:84 , srv2041514520:84 , srv2047481443:84 , srv2084516709:84 , srv2098564469:84 , srv209993363:84 , srv2117896849:84 , srv2124074727:84 , srv2146402675:84 , srv235477778:84 , srv241947280:84 , srv25137912:84 , srv289667038:84 , srv307706483:84 , srv321145497:84 , srv329302199:84 , srv352106251:84 , srv35417923:84 , srv372365566:84 , srv390501539:84 , srv392276979:84 , srv419234786:84 , srv43596680:84 , srv44176211:84 , srv561753023:84 , srv595425874:84 , srv595876929:84 , srv606629776:84 , srv612605341:84 , srv643056076:84 , srv65143754:84 , srv674692248:84 , srv678015011:84 , srv690484083:84 , srv707271281:84 , srv711521602:84 , srv733561716:84 , srv73841511:84 , srv743790733:84 , srv755763155:84 , srv759363567:84 , srv791076936:84 , srv868374390:84 , srv888481798:84 , srv967980869:84 , srv971753588:84 , srv991527456:84 , srv1670100484:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-08-23 02:53:29,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-23 02:53:29,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv733561716=89, srv390501539=72, srv1515187950=27, srv1798708615=45, srv743790733=91, srv1099637395=8, srv2014128489=52, srv307706483=66, srv1524859000=30, srv595876929=79, srv1392262761=20, srv2098564469=57, srv1441725338=23, srv1094119809=6, srv1902184167=48, srv1075283504=4, srv674692248=84, srv1213520811=17, srv612605341=81, srv971753588=98, srv419234786=74, srv1763378078=44, srv1401805386=21, srv44176211=76, srv2084516709=56, srv2117896849=59, srv791076936=94, srv2124074727=60, srv1067545616=2, srv241947280=63, srv606629776=80, srv759363567=93, srv1942054986=50, srv372365566=71, srv1516630381=28, srv20363050=53, srv1682881721=40, srv35417923=70, srv595425874=78, srv209993363=58, srv321145497=67, srv1018822835=0, srv289667038=65, srv1613607997=33, srv1121782692=10, srv643056076=82, srv2146402675=61, srv1130795841=12, srv888481798=96, srv1578064349=32, srv1691062550=41, srv1055429117=1, srv1194436131=15, srv1390577757=19, srv1101025502=9, srv1478477389=26, srv1524216183=29, srv65143754=83, srv1449102055=24, srv967980869=97, srv73841511=90, srv991527456=99, srv392276979=73, srv2047481443=55, srv1864728118=47, srv25137912=64, srv1071460766=3, srv1265970916=18, srv329302199=68, srv1535088183=31, srv1637079397=34, srv1122734407=11, srv1424312998=22, srv1919674422=49, srv1167213891=14, srv1675406261=39, srv561753023=77, srv1715588858=43, srv1670100484=37, srv1643681157=36, srv2041514520=54, srv707271281=87, srv1196117367=16, srv43596680=75, srv1855940610=46, srv690484083=86, srv1094863227=7, srv1671302576=38, srv352106251=69, srv711521602=88, srv678015011=85, srv1697481338=42, srv1454838615=25, srv868374390=95, srv1642254832=35, srv1970965449=51, srv755763155=92, srv1087380909=5, srv115228089=13, srv235477778=62} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-23 02:53:29,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-23 02:53:29,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-23 02:53:29,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-23 02:53:29,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-23 02:53:29,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-23 02:53:29,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-23 02:53:29,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-23 02:53:29,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-23 02:53:29,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-23 02:53:29,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-23 02:53:29,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-23 02:53:29,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-23 02:53:29,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-23 02:53:29,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-23 02:53:29,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-23 02:53:29,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-23 02:53:29,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-23 02:53:29,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-23 02:53:29,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-23 02:53:29,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-23 02:53:29,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-23 02:53:29,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-23 02:53:29,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-23 02:53:29,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-23 02:53:29,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-23 02:53:29,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-23 02:53:29,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-23 02:53:29,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-23 02:53:29,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-23 02:53:29,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-23 02:53:29,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-23 02:53:29,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-23 02:53:29,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-23 02:53:29,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-23 02:53:29,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-23 02:53:29,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-23 02:53:29,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-23 02:53:29,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-23 02:53:29,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-23 02:53:29,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-23 02:53:29,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-23 02:53:29,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-23 02:53:29,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-23 02:53:29,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-23 02:53:29,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-23 02:53:29,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-23 02:53:29,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-23 02:53:29,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-23 02:53:29,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-23 02:53:29,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-23 02:53:29,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-23 02:53:29,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-23 02:53:29,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-23 02:53:29,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-23 02:53:29,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-23 02:53:29,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-23 02:53:29,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-23 02:53:29,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-23 02:53:29,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-23 02:53:29,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-23 02:53:29,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-23 02:53:29,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-23 02:53:29,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-23 02:53:29,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-23 02:53:29,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-23 02:53:29,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-23 02:53:29,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-23 02:53:29,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-23 02:53:29,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-23 02:53:29,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-23 02:53:29,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-23 02:53:29,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-23 02:53:29,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-23 02:53:29,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-23 02:53:29,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-23 02:53:29,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-23 02:53:29,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-23 02:53:29,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-23 02:53:29,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-23 02:53:29,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-23 02:53:29,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-23 02:53:29,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-23 02:53:29,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-23 02:53:29,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-23 02:53:29,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-23 02:53:29,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-23 02:53:29,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-23 02:53:29,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-23 02:53:29,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-23 02:53:29,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-23 02:53:29,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-23 02:53:29,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-23 02:53:29,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-23 02:53:29,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-23 02:53:29,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-23 02:53:29,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-23 02:53:29,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-23 02:53:29,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-23 02:53:29,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-23 02:53:29,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-23 02:53:29,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-23 02:53:29,806 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-23 02:53:29,806 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-23 02:53:29,806 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-23 02:53:30,246 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 733 ms to try 5000 different iterations.  Found a solution that moves 4720 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17648165733679866. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040197479848962624, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05741924500522187, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5244444444444445, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040197479848962624, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1715, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23991666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-23 02:53:30,262 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv321145497:80 , srv971753588:80 , srv115228089:81 , srv235477778:81 , srv419234786:81 , srv991527456:81 , srv1265970916:82 , srv707271281:82 , srv1715588858:83 , srv1864728118:83 , srv2047481443:83 , srv329302199:83 , srv1087380909:84 , srv1196117367:84 , srv1392262761:84 , srv1516630381:84 , srv1691062550:84 , srv1855940610:84 , srv2146402675:84 , srv241947280:84 , srv65143754:84 , srv1094119809:85 , srv1094863227:85 , srv1390577757:85 , srv1478477389:85 , srv1524859000:85 , srv1763378078:85 , srv1919674422:85 , srv20363050:85 , srv2124074727:85 , srv352106251:85 , srv372365566:85 , srv390501539:85 , srv606629776:85 , srv612605341:85 , srv755763155:85 , srv1121782692:86 , srv1122734407:86 , srv1194436131:86 , srv1424312998:86 , srv1441725338:86 , srv1578064349:86 , srv1643681157:86 , srv1942054986:86 , srv2014128489:86 , srv209993363:86 , srv2117896849:86 , srv289667038:86 , srv44176211:86 , srv595876929:86 , srv643056076:86 , srv674692248:86 , srv759363567:86 , srv1018822835:87 , srv1167213891:87 , srv1213520811:87 , srv1524216183:87 , srv1671302576:87 , srv1682881721:87 , srv2084516709:87 , srv25137912:87 , srv43596680:87 , srv690484083:87 , srv733561716:87 , srv73841511:87 , srv1055429117:88 , srv1075283504:88 , srv1130795841:88 , srv1401805386:88 , srv1449102055:88 , srv1454838615:88 , srv1515187950:88 , srv1535088183:88 , srv1675406261:88 , srv1798708615:88 , srv2041514520:88 , srv2098564469:88 , srv595425874:88 , srv678015011:88 , srv868374390:88 , srv888481798:88 , srv1099637395:89 , srv1101025502:89 , srv1637079397:89 , srv1642254832:89 , srv1902184167:89 , srv1970965449:89 , srv307706483:89 , srv392276979:89 , srv1067545616:90 , srv1071460766:90 , srv1697481338:90 , srv967980869:90 , srv35417923:91 , srv561753023:91 , srv711521602:91 , srv743790733:91 , srv791076936:91 , srv1613607997:101 , srv1670100484:447 }
2023-08-23 02:53:30,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-23 02:53:30,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv733561716=89, srv390501539=72, srv1515187950=27, srv1798708615=45, srv743790733=91, srv1099637395=8, srv2014128489=52, srv307706483=66, srv1524859000=30, srv595876929=79, srv1392262761=20, srv2098564469=57, srv1441725338=23, srv1094119809=6, srv1902184167=48, srv1075283504=4, srv674692248=84, srv1213520811=17, srv612605341=81, srv971753588=98, srv419234786=74, srv1763378078=44, srv1401805386=21, srv44176211=76, srv2084516709=56, srv2117896849=59, srv791076936=94, srv2124074727=60, srv1067545616=2, srv241947280=63, srv606629776=80, srv759363567=93, srv1942054986=50, srv372365566=71, srv1516630381=28, srv20363050=53, srv1682881721=40, srv35417923=70, srv595425874=78, srv209993363=58, srv321145497=67, srv1018822835=0, srv289667038=65, srv1613607997=33, srv1121782692=10, srv643056076=82, srv2146402675=61, srv1130795841=12, srv888481798=96, srv1578064349=32, srv1691062550=41, srv1055429117=1, srv1194436131=15, srv1390577757=19, srv1101025502=9, srv1478477389=26, srv1524216183=29, srv65143754=83, srv1449102055=24, srv967980869=97, srv73841511=90, srv991527456=99, srv392276979=73, srv2047481443=55, srv1864728118=47, srv25137912=64, srv1071460766=3, srv1265970916=18, srv329302199=68, srv1535088183=31, srv1637079397=34, srv1122734407=11, srv1424312998=22, srv1919674422=49, srv1167213891=14, srv1675406261=39, srv561753023=77, srv1715588858=43, srv1670100484=37, srv1643681157=36, srv2041514520=54, srv707271281=87, srv1196117367=16, srv43596680=75, srv1855940610=46, srv690484083=86, srv1094863227=7, srv1671302576=38, srv352106251=69, srv711521602=88, srv678015011=85, srv1697481338=42, srv1454838615=25, srv868374390=95, srv1642254832=35, srv1970965449=51, srv755763155=92, srv1087380909=5, srv115228089=13, srv235477778=62} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-23 02:53:30,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-23 02:53:30,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-23 02:53:30,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-23 02:53:30,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-23 02:53:30,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-23 02:53:30,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-23 02:53:30,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-23 02:53:30,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-23 02:53:30,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-23 02:53:30,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-23 02:53:30,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-23 02:53:30,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-23 02:53:30,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-23 02:53:30,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-23 02:53:30,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-23 02:53:30,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-23 02:53:30,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-23 02:53:30,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-23 02:53:30,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-23 02:53:30,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-23 02:53:30,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-23 02:53:30,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-23 02:53:30,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-23 02:53:30,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-23 02:53:30,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-23 02:53:30,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-23 02:53:30,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-23 02:53:30,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-23 02:53:30,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-23 02:53:30,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-23 02:53:30,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-23 02:53:30,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-23 02:53:30,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-23 02:53:30,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-23 02:53:30,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-23 02:53:30,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-23 02:53:30,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-23 02:53:30,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-23 02:53:30,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-23 02:53:30,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-23 02:53:30,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-23 02:53:30,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-23 02:53:30,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-23 02:53:30,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-23 02:53:30,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-23 02:53:30,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-23 02:53:30,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-23 02:53:30,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-23 02:53:30,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-23 02:53:30,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-23 02:53:30,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-23 02:53:30,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-23 02:53:30,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-23 02:53:30,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-23 02:53:30,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-23 02:53:30,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-23 02:53:30,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-23 02:53:30,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-23 02:53:30,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-23 02:53:30,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-23 02:53:30,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-23 02:53:30,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-23 02:53:30,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-23 02:53:30,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-23 02:53:30,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-23 02:53:30,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-23 02:53:30,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-23 02:53:30,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-23 02:53:30,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-23 02:53:30,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-23 02:53:30,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-23 02:53:30,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-23 02:53:30,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-23 02:53:30,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-23 02:53:30,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-23 02:53:30,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-23 02:53:30,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-23 02:53:30,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-23 02:53:30,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-23 02:53:30,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-23 02:53:30,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-23 02:53:30,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-23 02:53:30,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-23 02:53:30,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-23 02:53:30,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-23 02:53:30,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-23 02:53:30,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-23 02:53:30,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-23 02:53:30,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-23 02:53:30,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-23 02:53:30,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-23 02:53:30,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-23 02:53:30,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-23 02:53:30,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-23 02:53:30,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-23 02:53:30,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-23 02:53:30,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-23 02:53:30,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-23 02:53:30,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-23 02:53:30,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-23 02:53:30,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-23 02:53:30,591 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040197479848962624, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05741924500522187, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040197479848962624, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1715, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23991666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-23 02:53:30,591 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-23 02:53:30,592 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17644860719040373, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040197479848962624, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05741924500522187, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040197479848962624, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1715, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23991666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-23 02:53:30,975 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 642 ms to try 5000 different iterations.  Found a solution that moves 2582 regions; Going from a computed imbalance of 0.17644860719040373 to a new imbalance of 0.015556519538521163. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022076966343861613); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03663757646158757, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2868888888888889, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022076966343861613); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012333333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04625, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-23 02:53:30,986 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1613607997:72 , srv35417923:79 , srv1071460766:81 , srv1515187950:81 , srv307706483:81 , srv743790733:81 , srv1099637395:82 , srv1642254832:82 , srv1697481338:82 , srv1902184167:82 , srv1970965449:82 , srv2041514520:82 , srv392276979:82 , srv711521602:82 , srv791076936:82 , srv1067545616:83 , srv1101025502:83 , srv1213520811:83 , srv1535088183:83 , srv1675406261:83 , srv1798708615:83 , srv561753023:83 , srv595425874:83 , srv733561716:83 , srv759363567:83 , srv1055429117:84 , srv1075283504:84 , srv1130795841:84 , srv1454838615:84 , srv1524216183:84 , srv1637079397:84 , srv1643681157:84 , srv43596680:84 , srv44176211:84 , srv595876929:84 , srv888481798:84 , srv1018822835:85 , srv1424312998:85 , srv1478477389:85 , srv1671302576:85 , srv1763378078:85 , srv209993363:85 , srv390501539:85 , srv612605341:85 , srv678015011:85 , srv690484083:85 , srv73841511:85 , srv755763155:85 , srv967980869:85 , srv1121782692:86 , srv1401805386:86 , srv1449102055:86 , srv1524859000:86 , srv1682881721:86 , srv1942054986:86 , srv20363050:86 , srv2084516709:86 , srv2098564469:86 , srv241947280:86 , srv606629776:86 , srv868374390:86 , srv1087380909:87 , srv1094863227:87 , srv1167213891:87 , srv1194436131:87 , srv1390577757:87 , srv1441725338:87 , srv1516630381:87 , srv1919674422:87 , srv2014128489:87 , srv2047481443:87 , srv329302199:87 , srv643056076:87 , srv65143754:87 , srv674692248:87 , srv1122734407:88 , srv1578064349:88 , srv1691062550:88 , srv2146402675:88 , srv25137912:88 , srv419234786:88 , srv1094119809:89 , srv115228089:89 , srv1855940610:89 , srv1864728118:89 , srv2117896849:89 , srv2124074727:89 , srv352106251:89 , srv1265970916:90 , srv1715588858:90 , srv235477778:90 , srv289667038:90 , srv1196117367:91 , srv372365566:91 , srv971753588:91 , srv991527456:91 , srv1392262761:92 , srv707271281:92 , srv321145497:96 , srv1670100484:523 }
2023-08-23 02:53:30,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-23 02:53:30,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv733561716=89, srv390501539=72, srv1515187950=27, srv1798708615=45, srv743790733=91, srv1099637395=8, srv2014128489=52, srv307706483=66, srv1524859000=30, srv595876929=79, srv1392262761=20, srv2098564469=57, srv1441725338=23, srv1094119809=6, srv1902184167=48, srv1075283504=4, srv674692248=84, srv1213520811=17, srv612605341=81, srv971753588=98, srv419234786=74, srv1763378078=44, srv1401805386=21, srv44176211=76, srv2084516709=56, srv2117896849=59, srv791076936=94, srv2124074727=60, srv1067545616=2, srv241947280=63, srv606629776=80, srv759363567=93, srv1942054986=50, srv372365566=71, srv1516630381=28, srv20363050=53, srv1682881721=40, srv35417923=70, srv595425874=78, srv209993363=58, srv321145497=67, srv1018822835=0, srv289667038=65, srv1613607997=33, srv1121782692=10, srv643056076=82, srv2146402675=61, srv1130795841=12, srv888481798=96, srv1578064349=32, srv1691062550=41, srv1055429117=1, srv1194436131=15, srv1390577757=19, srv1101025502=9, srv1478477389=26, srv1524216183=29, srv65143754=83, srv1449102055=24, srv967980869=97, srv73841511=90, srv991527456=99, srv392276979=73, srv2047481443=55, srv1864728118=47, srv25137912=64, srv1071460766=3, srv1265970916=18, srv329302199=68, srv1535088183=31, srv1637079397=34, srv1122734407=11, srv1424312998=22, srv1919674422=49, srv1167213891=14, srv1675406261=39, srv561753023=77, srv1715588858=43, srv1670100484=37, srv1643681157=36, srv2041514520=54, srv707271281=87, srv1196117367=16, srv43596680=75, srv1855940610=46, srv690484083=86, srv1094863227=7, srv1671302576=38, srv352106251=69, srv711521602=88, srv678015011=85, srv1697481338=42, srv1454838615=25, srv868374390=95, srv1642254832=35, srv1970965449=51, srv755763155=92, srv1087380909=5, srv115228089=13, srv235477778=62} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-23 02:53:31,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-23 02:53:31,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-23 02:53:31,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-23 02:53:31,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-23 02:53:31,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-23 02:53:31,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-23 02:53:31,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-23 02:53:31,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-23 02:53:31,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-23 02:53:31,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-23 02:53:31,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-23 02:53:31,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-23 02:53:31,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-23 02:53:31,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-23 02:53:31,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-23 02:53:31,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-23 02:53:31,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-23 02:53:31,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-23 02:53:31,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-23 02:53:31,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-23 02:53:31,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-23 02:53:31,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-23 02:53:31,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-23 02:53:31,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-23 02:53:31,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-23 02:53:31,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-23 02:53:31,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-23 02:53:31,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-23 02:53:31,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-23 02:53:31,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-23 02:53:31,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-23 02:53:31,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-23 02:53:31,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-23 02:53:31,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-23 02:53:31,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-23 02:53:31,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-23 02:53:31,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-23 02:53:31,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-23 02:53:31,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-23 02:53:31,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-23 02:53:31,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-23 02:53:31,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-23 02:53:31,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-23 02:53:31,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-23 02:53:31,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-23 02:53:31,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-23 02:53:31,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-23 02:53:31,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-23 02:53:31,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-23 02:53:31,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-23 02:53:31,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-23 02:53:31,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-23 02:53:31,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-23 02:53:31,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-23 02:53:31,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-23 02:53:31,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-23 02:53:31,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-23 02:53:31,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-23 02:53:31,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-23 02:53:31,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-23 02:53:31,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-23 02:53:31,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-23 02:53:31,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-23 02:53:31,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-23 02:53:31,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-23 02:53:31,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-23 02:53:31,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-23 02:53:31,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-23 02:53:31,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-23 02:53:31,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-23 02:53:31,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-23 02:53:31,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-23 02:53:31,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-23 02:53:31,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-23 02:53:31,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-23 02:53:31,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-23 02:53:31,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-23 02:53:31,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-23 02:53:31,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-23 02:53:31,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-23 02:53:31,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-23 02:53:31,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-23 02:53:31,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-23 02:53:31,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-23 02:53:31,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-23 02:53:31,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-23 02:53:31,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-23 02:53:31,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-23 02:53:31,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-23 02:53:31,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-23 02:53:31,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-23 02:53:31,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-23 02:53:31,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-23 02:53:31,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-23 02:53:31,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-23 02:53:31,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-23 02:53:31,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-23 02:53:31,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-23 02:53:31,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-23 02:53:31,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-23 02:53:31,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-23 02:53:31,196 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022076966343861613); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03663757646158757, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022076966343861613); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012333333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04625, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-23 02:53:31,196 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-23 02:53:31,196 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.015538439988099182, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022076966343861613); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03663757646158757, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022076966343861613); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012333333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04625, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-23 02:53:31,599 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 578 ms to try 5000 different iterations.  Found a solution that moves 983 regions; Going from a computed imbalance of 0.015538439988099182 to a new imbalance of 7.028273598648952E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006442160669684254); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009380352942419313); MoveCostFunction : (multiplier=7.0, imbalance=0.10922222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006442160669684254); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-23 02:53:31,602 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1392262761:81 , srv289667038:81 , srv321145497:82 , srv1067545616:83 , srv1094119809:83 , srv1122734407:83 , srv1167213891:83 , srv1196117367:83 , srv1401805386:83 , srv1637079397:83 , srv2117896849:83 , srv2124074727:83 , srv25137912:83 , srv372365566:83 , srv561753023:83 , srv707271281:83 , srv967980869:83 , srv1018822835:84 , srv1075283504:84 , srv1130795841:84 , srv1194436131:84 , srv1449102055:84 , srv1578064349:84 , srv1613607997:84 , srv1682881721:84 , srv1919674422:84 , srv2014128489:84 , srv2084516709:84 , srv2098564469:84 , srv352106251:84 , srv674692248:84 , srv678015011:84 , srv711521602:84 , srv868374390:84 , srv1055429117:85 , srv1101025502:85 , srv1265970916:85 , srv1441725338:85 , srv1454838615:85 , srv1671302576:85 , srv1697481338:85 , srv1715588858:85 , srv1855940610:85 , srv1942054986:85 , srv2146402675:85 , srv73841511:85 , srv791076936:85 , srv1071460766:86 , srv1087380909:86 , srv1094863227:86 , srv1121782692:86 , srv1390577757:86 , srv1516630381:86 , srv1524216183:86 , srv1691062550:86 , srv1798708615:86 , srv1864728118:86 , srv1970965449:86 , srv20363050:86 , srv235477778:86 , srv43596680:86 , srv606629776:86 , srv643056076:86 , srv690484083:86 , srv743790733:86 , srv888481798:86 , srv971753588:86 , srv991527456:86 , srv1099637395:87 , srv115228089:87 , srv1213520811:87 , srv1424312998:87 , srv1478477389:87 , srv1524859000:87 , srv1535088183:87 , srv1675406261:87 , srv1763378078:87 , srv1902184167:87 , srv2041514520:87 , srv209993363:87 , srv241947280:87 , srv329302199:87 , srv35417923:87 , srv390501539:87 , srv392276979:87 , srv44176211:87 , srv595425874:87 , srv65143754:87 , srv733561716:87 , srv755763155:87 , srv1642254832:88 , srv2047481443:88 , srv419234786:88 , srv595876929:88 , srv612605341:88 , srv1515187950:89 , srv1643681157:89 , srv307706483:89 , srv759363567:89 , srv1670100484:545 }
2023-08-23 02:53:31,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-23 02:53:31,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv733561716=89, srv390501539=72, srv1515187950=27, srv1798708615=45, srv743790733=91, srv1099637395=8, srv2014128489=52, srv307706483=66, srv1524859000=30, srv595876929=79, srv1392262761=20, srv2098564469=57, srv1441725338=23, srv1094119809=6, srv1902184167=48, srv1075283504=4, srv674692248=84, srv1213520811=17, srv612605341=81, srv971753588=98, srv419234786=74, srv1763378078=44, srv1401805386=21, srv44176211=76, srv2084516709=56, srv2117896849=59, srv791076936=94, srv2124074727=60, srv1067545616=2, srv241947280=63, srv606629776=80, srv759363567=93, srv1942054986=50, srv372365566=71, srv1516630381=28, srv20363050=53, srv1682881721=40, srv35417923=70, srv595425874=78, srv209993363=58, srv321145497=67, srv1018822835=0, srv289667038=65, srv1613607997=33, srv1121782692=10, srv643056076=82, srv2146402675=61, srv1130795841=12, srv888481798=96, srv1578064349=32, srv1691062550=41, srv1055429117=1, srv1194436131=15, srv1390577757=19, srv1101025502=9, srv1478477389=26, srv1524216183=29, srv65143754=83, srv1449102055=24, srv967980869=97, srv73841511=90, srv991527456=99, srv392276979=73, srv2047481443=55, srv1864728118=47, srv25137912=64, srv1071460766=3, srv1265970916=18, srv329302199=68, srv1535088183=31, srv1637079397=34, srv1122734407=11, srv1424312998=22, srv1919674422=49, srv1167213891=14, srv1675406261=39, srv561753023=77, srv1715588858=43, srv1670100484=37, srv1643681157=36, srv2041514520=54, srv707271281=87, srv1196117367=16, srv43596680=75, srv1855940610=46, srv690484083=86, srv1094863227=7, srv1671302576=38, srv352106251=69, srv711521602=88, srv678015011=85, srv1697481338=42, srv1454838615=25, srv868374390=95, srv1642254832=35, srv1970965449=51, srv755763155=92, srv1087380909=5, srv115228089=13, srv235477778=62} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-23 02:53:31,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-23 02:53:31,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-23 02:53:31,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-23 02:53:31,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-23 02:53:31,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-23 02:53:31,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-23 02:53:31,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-23 02:53:31,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-23 02:53:31,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-23 02:53:31,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-23 02:53:31,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-23 02:53:31,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-23 02:53:31,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-23 02:53:31,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-23 02:53:31,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-23 02:53:31,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-23 02:53:31,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-23 02:53:31,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-23 02:53:31,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-23 02:53:31,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-23 02:53:31,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-23 02:53:31,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-23 02:53:31,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-23 02:53:31,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-23 02:53:31,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-23 02:53:31,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-23 02:53:31,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-23 02:53:31,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-23 02:53:31,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-23 02:53:31,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-23 02:53:31,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-23 02:53:31,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-23 02:53:31,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-23 02:53:31,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-23 02:53:31,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-23 02:53:31,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-23 02:53:31,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-23 02:53:31,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-23 02:53:31,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-23 02:53:31,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-23 02:53:31,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-23 02:53:31,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-23 02:53:31,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-23 02:53:31,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-23 02:53:31,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-23 02:53:31,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-23 02:53:31,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-23 02:53:31,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-23 02:53:31,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-23 02:53:31,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-23 02:53:31,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-23 02:53:31,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-23 02:53:31,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-23 02:53:31,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-23 02:53:31,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-23 02:53:31,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-23 02:53:31,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-23 02:53:31,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-23 02:53:31,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-23 02:53:31,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-23 02:53:31,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-23 02:53:31,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-23 02:53:31,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-23 02:53:31,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-23 02:53:31,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-23 02:53:31,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-23 02:53:31,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-23 02:53:31,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-23 02:53:31,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-23 02:53:31,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-23 02:53:31,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-23 02:53:31,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-23 02:53:31,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-23 02:53:31,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-23 02:53:31,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-23 02:53:31,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-23 02:53:31,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-23 02:53:31,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-23 02:53:31,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-23 02:53:31,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-23 02:53:31,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-23 02:53:31,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-23 02:53:31,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-23 02:53:31,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-23 02:53:31,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-23 02:53:31,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-23 02:53:31,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-23 02:53:31,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-23 02:53:31,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-23 02:53:31,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-23 02:53:31,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-23 02:53:31,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-23 02:53:31,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-23 02:53:31,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-23 02:53:31,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-23 02:53:31,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-23 02:53:31,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-23 02:53:31,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-23 02:53:31,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-23 02:53:31,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-23 02:53:31,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-23 02:53:31,817 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006442160669684254); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009380352942419313); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006442160669684254); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-23 02:53:31,818 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-23 02:53:31,818 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=6.959442467491686E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006442160669684254); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009380352942419313); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006442160669684254); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-23 02:53:32,248 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 578 ms to try 5000 different iterations.  Found a solution that moves 230 regions; Going from a computed imbalance of 6.959442467491686E-4 to a new imbalance of 1.2152206630888418E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.025555555555555557, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-23 02:53:32,249 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1642254832:83 , srv1643681157:83 , srv307706483:83 , srv643056076:83 , srv759363567:83 , srv1099637395:84 , srv1121782692:84 , srv1167213891:84 , srv1196117367:84 , srv1213520811:84 , srv1390577757:84 , srv1424312998:84 , srv1441725338:84 , srv1449102055:84 , srv1515187950:84 , srv1524859000:84 , srv1535088183:84 , srv1578064349:84 , srv1613607997:84 , srv1675406261:84 , srv1715588858:84 , srv1864728118:84 , srv1902184167:84 , srv20363050:84 , srv2047481443:84 , srv2084516709:84 , srv2098564469:84 , srv209993363:84 , srv2117896849:84 , srv2146402675:84 , srv25137912:84 , srv321145497:84 , srv352106251:84 , srv372365566:84 , srv392276979:84 , srv561753023:84 , srv595425874:84 , srv595876929:84 , srv612605341:84 , srv65143754:84 , srv690484083:84 , srv743790733:84 , srv791076936:84 , srv868374390:84 , srv888481798:84 , srv967980869:84 , srv991527456:84 , srv1055429117:85 , srv1067545616:85 , srv1071460766:85 , srv1087380909:85 , srv1094119809:85 , srv1094863227:85 , srv1101025502:85 , srv1122734407:85 , srv115228089:85 , srv1194436131:85 , srv1392262761:85 , srv1401805386:85 , srv1454838615:85 , srv1478477389:85 , srv1516630381:85 , srv1524216183:85 , srv1671302576:85 , srv1691062550:85 , srv1697481338:85 , srv1763378078:85 , srv1798708615:85 , srv1855940610:85 , srv1942054986:85 , srv1970965449:85 , srv2014128489:85 , srv2041514520:85 , srv2124074727:85 , srv235477778:85 , srv241947280:85 , srv289667038:85 , srv329302199:85 , srv390501539:85 , srv419234786:85 , srv43596680:85 , srv44176211:85 , srv606629776:85 , srv674692248:85 , srv678015011:85 , srv707271281:85 , srv733561716:85 , srv73841511:85 , srv755763155:85 , srv1018822835:86 , srv1075283504:86 , srv1130795841:86 , srv1265970916:86 , srv1637079397:86 , srv1682881721:86 , srv1919674422:86 , srv35417923:86 , srv711521602:86 , srv971753588:86 , srv1670100484:627 }
2023-08-23 02:53:32,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-23 02:53:32,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv733561716=89, srv390501539=72, srv1515187950=27, srv1798708615=45, srv743790733=91, srv1099637395=8, srv2014128489=52, srv307706483=66, srv1524859000=30, srv595876929=79, srv1392262761=20, srv2098564469=57, srv1441725338=23, srv1094119809=6, srv1902184167=48, srv1075283504=4, srv674692248=84, srv1213520811=17, srv612605341=81, srv971753588=98, srv419234786=74, srv1763378078=44, srv1401805386=21, srv44176211=76, srv2084516709=56, srv2117896849=59, srv791076936=94, srv2124074727=60, srv1067545616=2, srv241947280=63, srv606629776=80, srv759363567=93, srv1942054986=50, srv372365566=71, srv1516630381=28, srv20363050=53, srv1682881721=40, srv35417923=70, srv595425874=78, srv209993363=58, srv321145497=67, srv1018822835=0, srv289667038=65, srv1613607997=33, srv1121782692=10, srv643056076=82, srv2146402675=61, srv1130795841=12, srv888481798=96, srv1578064349=32, srv1691062550=41, srv1055429117=1, srv1194436131=15, srv1390577757=19, srv1101025502=9, srv1478477389=26, srv1524216183=29, srv65143754=83, srv1449102055=24, srv967980869=97, srv73841511=90, srv991527456=99, srv392276979=73, srv2047481443=55, srv1864728118=47, srv25137912=64, srv1071460766=3, srv1265970916=18, srv329302199=68, srv1535088183=31, srv1637079397=34, srv1122734407=11, srv1424312998=22, srv1919674422=49, srv1167213891=14, srv1675406261=39, srv561753023=77, srv1715588858=43, srv1670100484=37, srv1643681157=36, srv2041514520=54, srv707271281=87, srv1196117367=16, srv43596680=75, srv1855940610=46, srv690484083=86, srv1094863227=7, srv1671302576=38, srv352106251=69, srv711521602=88, srv678015011=85, srv1697481338=42, srv1454838615=25, srv868374390=95, srv1642254832=35, srv1970965449=51, srv755763155=92, srv1087380909=5, srv115228089=13, srv235477778=62} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-23 02:53:32,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-23 02:53:32,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-23 02:53:32,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-23 02:53:32,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-23 02:53:32,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-23 02:53:32,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-23 02:53:32,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-23 02:53:32,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-23 02:53:32,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-23 02:53:32,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-23 02:53:32,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-23 02:53:32,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-23 02:53:32,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-23 02:53:32,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-23 02:53:32,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-23 02:53:32,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-23 02:53:32,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-23 02:53:32,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-23 02:53:32,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-23 02:53:32,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-23 02:53:32,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-23 02:53:32,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-23 02:53:32,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-23 02:53:32,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-23 02:53:32,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-23 02:53:32,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-23 02:53:32,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-23 02:53:32,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-23 02:53:32,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-23 02:53:32,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-23 02:53:32,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-23 02:53:32,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-23 02:53:32,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-23 02:53:32,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-23 02:53:32,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-23 02:53:32,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-23 02:53:32,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-23 02:53:32,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-23 02:53:32,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-23 02:53:32,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-23 02:53:32,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-23 02:53:32,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-23 02:53:32,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-23 02:53:32,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-23 02:53:32,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-23 02:53:32,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-23 02:53:32,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-23 02:53:32,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-23 02:53:32,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-23 02:53:32,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-23 02:53:32,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-23 02:53:32,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-23 02:53:32,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-23 02:53:32,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-23 02:53:32,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-23 02:53:32,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-23 02:53:32,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-23 02:53:32,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-23 02:53:32,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-23 02:53:32,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-23 02:53:32,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-23 02:53:32,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-23 02:53:32,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-23 02:53:32,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-23 02:53:32,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-23 02:53:32,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-23 02:53:32,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-23 02:53:32,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-23 02:53:32,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-23 02:53:32,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-23 02:53:32,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-23 02:53:32,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-23 02:53:32,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-23 02:53:32,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-23 02:53:32,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-23 02:53:32,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-23 02:53:32,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-23 02:53:32,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-23 02:53:32,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-23 02:53:32,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-23 02:53:32,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-23 02:53:32,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-23 02:53:32,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-23 02:53:32,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-23 02:53:32,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-23 02:53:32,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-23 02:53:32,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-23 02:53:32,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-23 02:53:32,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-23 02:53:32,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-23 02:53:32,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-23 02:53:32,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-23 02:53:32,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-23 02:53:32,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-23 02:53:32,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-23 02:53:32,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-23 02:53:32,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-23 02:53:32,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-23 02:53:32,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-23 02:53:32,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-23 02:53:32,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-23 02:53:32,442 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.0541712209100929E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-23 02:53:32,442 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1642254832:83 , srv1643681157:83 , srv307706483:83 , srv643056076:83 , srv759363567:83 , srv1099637395:84 , srv1121782692:84 , srv1167213891:84 , srv1196117367:84 , srv1213520811:84 , srv1390577757:84 , srv1424312998:84 , srv1441725338:84 , srv1449102055:84 , srv1515187950:84 , srv1524859000:84 , srv1535088183:84 , srv1578064349:84 , srv1613607997:84 , srv1675406261:84 , srv1715588858:84 , srv1864728118:84 , srv1902184167:84 , srv20363050:84 , srv2047481443:84 , srv2084516709:84 , srv2098564469:84 , srv209993363:84 , srv2117896849:84 , srv2146402675:84 , srv25137912:84 , srv321145497:84 , srv352106251:84 , srv372365566:84 , srv392276979:84 , srv561753023:84 , srv595425874:84 , srv595876929:84 , srv612605341:84 , srv65143754:84 , srv690484083:84 , srv743790733:84 , srv791076936:84 , srv868374390:84 , srv888481798:84 , srv967980869:84 , srv991527456:84 , srv1055429117:85 , srv1067545616:85 , srv1071460766:85 , srv1087380909:85 , srv1094119809:85 , srv1094863227:85 , srv1101025502:85 , srv1122734407:85 , srv115228089:85 , srv1194436131:85 , srv1392262761:85 , srv1401805386:85 , srv1454838615:85 , srv1478477389:85 , srv1516630381:85 , srv1524216183:85 , srv1671302576:85 , srv1691062550:85 , srv1697481338:85 , srv1763378078:85 , srv1798708615:85 , srv1855940610:85 , srv1942054986:85 , srv1970965449:85 , srv2014128489:85 , srv2041514520:85 , srv2124074727:85 , srv235477778:85 , srv241947280:85 , srv289667038:85 , srv329302199:85 , srv390501539:85 , srv419234786:85 , srv43596680:85 , srv44176211:85 , srv606629776:85 , srv674692248:85 , srv678015011:85 , srv707271281:85 , srv733561716:85 , srv73841511:85 , srv755763155:85 , srv1018822835:86 , srv1075283504:86 , srv1130795841:86 , srv1265970916:86 , srv1637079397:86 , srv1682881721:86 , srv1919674422:86 , srv35417923:86 , srv711521602:86 , srv971753588:86 , srv1670100484:627 }
2023-08-23 02:53:32,482 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=294 (was 294), ProcessCount=171 (was 171), AvailableMemoryMB=11483 (was 12367)
2023-08-23 02:53:32,492 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=294, ProcessCount=171, AvailableMemoryMB=11481
2023-08-23 02:53:32,493 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-23 02:53:32,493 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-08-23 02:53:32,506 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1052903923:84 , srv1058535943:84 , srv1069172975:84 , srv1225348136:84 , srv1238083638:84 , srv129928925:84 , srv135616652:84 , srv1456682774:84 , srv146147571:84 , srv1473198956:84 , srv1590274885:84 , srv1693988497:84 , srv1828238945:84 , srv1889656178:84 , srv1910619110:84 , srv1980853686:84 , srv2016642254:84 , srv2034209444:84 , srv2042762978:84 , srv226935300:84 , srv241430278:84 , srv496147213:84 , srv580183867:84 , srv663086811:84 , srv666417776:84 , srv704744193:84 , srv805827319:84 , srv905429549:84 , srv933971334:84 , srv1915482522:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-08-23 02:53:32,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-23 02:53:32,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv580183867=23, srv135616652=6, srv226935300=20, srv1456682774=7, srv129928925=5, srv146147571=8, srv1238083638=4, srv1915482522=15, srv1910619110=14, srv2042762978=19, srv1980853686=16, srv1828238945=12, srv666417776=25, srv1069172975=2, srv1058535943=1, srv1590274885=10, srv663086811=24, srv2016642254=17, srv704744193=26, srv805827319=27, srv1225348136=3, srv1052903923=0, srv1889656178=13, srv2034209444=18, srv1473198956=9, srv905429549=28, srv933971334=29, srv241430278=21, srv496147213=22, srv1693988497=11} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-23 02:53:32,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-23 02:53:32,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-23 02:53:32,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-23 02:53:32,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-23 02:53:32,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-23 02:53:32,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-23 02:53:32,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-23 02:53:32,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-23 02:53:32,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-23 02:53:32,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-23 02:53:32,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-23 02:53:32,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-23 02:53:32,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-23 02:53:32,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-23 02:53:32,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-23 02:53:32,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-23 02:53:32,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-23 02:53:32,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-23 02:53:32,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-23 02:53:32,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-23 02:53:32,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-23 02:53:32,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-23 02:53:32,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-23 02:53:32,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-23 02:53:32,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-23 02:53:32,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-23 02:53:32,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-23 02:53:32,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-23 02:53:32,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-23 02:53:32,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-23 02:53:32,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-08-23 02:53:32,536 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0792926115568013, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-23 02:53:32,536 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912442891694154, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0792926115568013, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-08-23 02:55:32,523 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5576152 different iterations.  Found a solution that moves 1856 regions; Going from a computed imbalance of 0.9912442891694154 to a new imbalance of 4.3319965896196796E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6874074074074074, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-23 02:55:32,531 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1052903923:90 , srv1058535943:90 , srv1069172975:90 , srv1225348136:90 , srv1238083638:90 , srv129928925:90 , srv135616652:90 , srv1456682774:90 , srv146147571:90 , srv1473198956:90 , srv1590274885:90 , srv1693988497:90 , srv1828238945:90 , srv1889656178:90 , srv1910619110:90 , srv1915482522:90 , srv1980853686:90 , srv2016642254:90 , srv2034209444:90 , srv2042762978:90 , srv226935300:90 , srv241430278:90 , srv496147213:90 , srv580183867:90 , srv663086811:90 , srv666417776:90 , srv704744193:90 , srv805827319:90 , srv905429549:90 , srv933971334:90 }
2023-08-23 02:55:32,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-23 02:55:32,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv580183867=23, srv135616652=6, srv226935300=20, srv1456682774=7, srv129928925=5, srv146147571=8, srv1238083638=4, srv1915482522=15, srv1910619110=14, srv2042762978=19, srv1980853686=16, srv1828238945=12, srv666417776=25, srv1069172975=2, srv1058535943=1, srv1590274885=10, srv663086811=24, srv2016642254=17, srv704744193=26, srv805827319=27, srv1225348136=3, srv1052903923=0, srv1889656178=13, srv2034209444=18, srv1473198956=9, srv905429549=28, srv933971334=29, srv241430278=21, srv496147213=22, srv1693988497=11} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-23 02:55:32,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-23 02:55:32,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-23 02:55:32,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-23 02:55:32,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-23 02:55:32,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-23 02:55:32,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-23 02:55:32,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-23 02:55:32,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-23 02:55:32,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-23 02:55:32,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-23 02:55:32,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-23 02:55:32,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-23 02:55:32,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-23 02:55:32,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-23 02:55:32,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-23 02:55:32,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-23 02:55:32,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-23 02:55:32,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-23 02:55:32,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-23 02:55:32,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-23 02:55:32,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-23 02:55:32,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-23 02:55:32,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-23 02:55:32,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-23 02:55:32,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-23 02:55:32,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-23 02:55:32,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-23 02:55:32,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-23 02:55:32,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-23 02:55:32,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-23 02:55:32,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-08-23 02:55:32,565 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-23 02:55:32,565 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1052903923:90 , srv1058535943:90 , srv1069172975:90 , srv1225348136:90 , srv1238083638:90 , srv129928925:90 , srv135616652:90 , srv1456682774:90 , srv146147571:90 , srv1473198956:90 , srv1590274885:90 , srv1693988497:90 , srv1828238945:90 , srv1889656178:90 , srv1910619110:90 , srv1915482522:90 , srv1980853686:90 , srv2016642254:90 , srv2034209444:90 , srv2042762978:90 , srv226935300:90 , srv241430278:90 , srv496147213:90 , srv580183867:90 , srv663086811:90 , srv666417776:90 , srv704744193:90 , srv805827319:90 , srv905429549:90 , srv933971334:90 }
2023-08-23 02:55:32,582 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=244 (was 294), ProcessCount=169 (was 171), AvailableMemoryMB=11379 (was 11481)