2023-04-23 08:55:37,159 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-04-23 08:55:37,522 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-04-23 08:55:37,655 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-04-23 08:55:37,737 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-23 08:55:37,746 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-04-23 08:55:37,761 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=455, ProcessCount=169, AvailableMemoryMB=13068
2023-04-23 08:55:37,763 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-23 08:55:37,764 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-04-23 08:55:37,926 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1015936688:84 , srv1029473835:84 , srv1138267611:84 , srv1154535672:84 , srv1227366938:84 , srv1245294538:84 , srv1257427698:84 , srv1260944144:84 , srv1306952321:84 , srv1333830018:84 , srv1353540096:84 , srv1392890793:84 , srv1417594755:84 , srv1428901854:84 , srv1445798510:84 , srv1464658439:84 , srv1475941883:84 , srv1478489484:84 , srv1482708482:84 , srv149429439:84 , srv1528679002:84 , srv1544044432:84 , srv1554956033:84 , srv1599363363:84 , srv1607715064:84 , srv1618408670:84 , srv1657321837:84 , srv1693000573:84 , srv1768696740:84 , srv1788563628:84 , srv1878978779:84 , srv1881959713:84 , srv1921091157:84 , srv1921588734:84 , srv1938161541:84 , srv1970144413:84 , srv1975499969:84 , srv1976453088:84 , srv1979172918:84 , srv1994834283:84 , srv2003072574:84 , srv2020383804:84 , srv2056396230:84 , srv2082007281:84 , srv2092135987:84 , srv253642703:84 , srv297862391:84 , srv30410366:84 , srv314534318:84 , srv332203072:84 , srv352840004:84 , srv353267356:84 , srv357615321:84 , srv390138966:84 , srv413357515:84 , srv446448345:84 , srv453906192:84 , srv469169741:84 , srv478475543:84 , srv489579947:84 , srv495211703:84 , srv507094004:84 , srv528935375:84 , srv545060236:84 , srv557445629:84 , srv559313821:84 , srv567933417:84 , srv623205500:84 , srv64746896:84 , srv664902431:84 , srv66856882:84 , srv673575094:84 , srv676855412:84 , srv681286211:84 , srv681666796:84 , srv737391917:84 , srv753789424:84 , srv755899798:84 , srv764535174:84 , srv766776048:84 , srv767751685:84 , srv792859801:84 , srv797472118:84 , srv800695755:84 , srv811994333:84 , srv813316128:84 , srv843805300:84 , srv848347606:84 , srv849649275:84 , srv875649862:84 , srv895075756:84 , srv904138546:84 , srv90592646:84 , srv907432331:84 , srv921652753:84 , srv966349708:84 , srv976077466:84 , srv985170869:84 , srv992248108:84 , srv1904030274:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-04-23 08:55:37,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-23 08:55:37,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv453906192=57, srv1482708482=18, srv1445798510=14, srv1428901854=13, srv1693000573=27, srv30410366=48, srv976077466=97, srv1975499969=37, srv557445629=65, srv1306952321=8, srv1333830018=9, srv800695755=84, srv1788563628=29, srv813316128=86, srv848347606=88, srv664902431=70, srv843805300=87, srv673575094=72, srv1475941883=16, srv149429439=19, srv1878978779=30, srv1904030274=32, srv985170869=98, srv1554956033=22, srv446448345=56, srv1353540096=10, srv1976453088=38, srv766776048=80, srv1544044432=21, srv1245294538=5, srv676855412=73, srv907432331=94, srv567933417=67, srv1257427698=6, srv1478489484=17, srv1138267611=2, srv1260944144=7, srv1015936688=0, srv992248108=99, srv1392890793=11, srv1599363363=23, srv390138966=54, srv2082007281=44, srv1227366938=4, srv332203072=50, srv1921091157=33, srv1881959713=31, srv1970144413=36, srv1921588734=34, srv1528679002=20, srv753789424=77, srv253642703=46, srv1417594755=12, srv507094004=62, srv1979172918=39, srv478475543=59, srv811994333=85, srv1607715064=24, srv966349708=96, srv1994834283=40, srv2020383804=42, srv623205500=68, srv2003072574=41, srv792859801=82, srv755899798=78, srv681286211=74, srv1029473835=1, srv469169741=58, srv904138546=92, srv764535174=79, srv1768696740=28, srv2092135987=45, srv352840004=51, srv797472118=83, srv545060236=64, srv297862391=47, srv1618408670=25, srv875649862=90, srv895075756=91, srv413357515=55, srv90592646=93, srv681666796=75, srv849649275=89, srv1154535672=3, srv2056396230=43, srv314534318=49, srv528935375=63, srv1938161541=35, srv559313821=66, srv489579947=60, srv64746896=69, srv767751685=81, srv921652753=95, srv495211703=61, srv66856882=71, srv357615321=53, srv353267356=52, srv1657321837=26, srv1464658439=15, srv737391917=76} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-23 08:55:38,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-23 08:55:38,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-23 08:55:38,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-23 08:55:38,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-23 08:55:38,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-23 08:55:38,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-23 08:55:38,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-23 08:55:38,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-23 08:55:38,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-23 08:55:38,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-23 08:55:38,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-23 08:55:38,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-23 08:55:38,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-23 08:55:38,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-23 08:55:38,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-23 08:55:38,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-23 08:55:38,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-23 08:55:38,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-23 08:55:38,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-23 08:55:38,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-23 08:55:38,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-23 08:55:38,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-23 08:55:38,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-23 08:55:38,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-23 08:55:38,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-23 08:55:38,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-23 08:55:38,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-23 08:55:38,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-23 08:55:38,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-23 08:55:38,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-23 08:55:38,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-23 08:55:38,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-23 08:55:38,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-23 08:55:38,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-23 08:55:38,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-23 08:55:38,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-23 08:55:38,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-23 08:55:38,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-23 08:55:38,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-23 08:55:38,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-23 08:55:38,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-23 08:55:38,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-23 08:55:38,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-23 08:55:38,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-23 08:55:38,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-23 08:55:38,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-23 08:55:38,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-23 08:55:38,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-23 08:55:38,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-23 08:55:38,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-23 08:55:38,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-23 08:55:38,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-23 08:55:38,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-23 08:55:38,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-23 08:55:38,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-23 08:55:38,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-23 08:55:38,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-23 08:55:38,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-23 08:55:38,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-23 08:55:38,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-23 08:55:38,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-23 08:55:38,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-23 08:55:38,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-23 08:55:38,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-23 08:55:38,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-23 08:55:38,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-23 08:55:38,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-23 08:55:38,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-23 08:55:38,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-23 08:55:38,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-23 08:55:38,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-23 08:55:38,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-23 08:55:38,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-23 08:55:38,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-23 08:55:38,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-23 08:55:38,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-23 08:55:38,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-23 08:55:38,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-23 08:55:38,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-23 08:55:38,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-23 08:55:38,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-23 08:55:38,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-23 08:55:38,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-23 08:55:38,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-23 08:55:38,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-23 08:55:38,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-23 08:55:38,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-23 08:55:38,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-23 08:55:38,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-23 08:55:38,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-23 08:55:38,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-23 08:55:38,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-23 08:55:38,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-23 08:55:38,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-23 08:55:38,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-23 08:55:38,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-23 08:55:38,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-23 08:55:38,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-23 08:55:38,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-23 08:55:38,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-23 08:55:38,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-23 08:55:38,516 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-23 08:55:38,516 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-23 08:55:38,516 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-23 08:55:39,370 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 1267 ms to try 5000 different iterations.  Found a solution that moves 4744 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17116325832867718. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04062456653197621, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0537546490041772, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5271111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04062456653197621, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.16575, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2385, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-23 08:55:39,404 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv737391917:81 , srv446448345:82 , srv453906192:82 , srv1154535672:83 , srv1257427698:83 , srv1333830018:83 , srv1544044432:83 , srv1657321837:83 , srv390138966:83 , srv792859801:83 , srv843805300:83 , srv1788563628:84 , srv1975499969:84 , srv1979172918:84 , srv2020383804:84 , srv352840004:84 , srv413357515:84 , srv478475543:84 , srv567933417:84 , srv64746896:84 , srv755899798:84 , srv766776048:84 , srv813316128:84 , srv966349708:84 , srv1245294538:85 , srv1392890793:85 , srv149429439:85 , srv1528679002:85 , srv1554956033:85 , srv1599363363:85 , srv1970144413:85 , srv1976453088:85 , srv314534318:85 , srv357615321:85 , srv528935375:85 , srv559313821:85 , srv764535174:85 , srv797472118:85 , srv875649862:85 , srv907432331:85 , srv1029473835:86 , srv1138267611:86 , srv1227366938:86 , srv1353540096:86 , srv1464658439:86 , srv1768696740:86 , srv1921091157:86 , srv1921588734:86 , srv1994834283:86 , srv2003072574:86 , srv2092135987:86 , srv253642703:86 , srv495211703:86 , srv507094004:86 , srv545060236:86 , srv623205500:86 , srv664902431:86 , srv767751685:86 , srv921652753:86 , srv976077466:86 , srv1881959713:87 , srv2056396230:87 , srv2082007281:87 , srv332203072:87 , srv557445629:87 , srv66856882:87 , srv673575094:87 , srv681666796:87 , srv849649275:87 , srv904138546:87 , srv985170869:87 , srv1428901854:88 , srv1482708482:88 , srv1607715064:88 , srv1618408670:88 , srv1938161541:88 , srv297862391:88 , srv30410366:88 , srv353267356:88 , srv489579947:88 , srv676855412:88 , srv753789424:88 , srv90592646:88 , srv1417594755:89 , srv1475941883:89 , srv1878978779:89 , srv848347606:89 , srv1306952321:90 , srv1445798510:90 , srv800695755:90 , srv1478489484:91 , srv1693000573:91 , srv811994333:91 , srv895075756:91 , srv992248108:91 , srv1260944144:92 , srv469169741:92 , srv681286211:92 , srv1015936688:98 , srv1904030274:451 }
2023-04-23 08:55:39,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-23 08:55:39,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv453906192=57, srv1482708482=18, srv1445798510=14, srv1428901854=13, srv1693000573=27, srv30410366=48, srv976077466=97, srv1975499969=37, srv557445629=65, srv1306952321=8, srv1333830018=9, srv800695755=84, srv1788563628=29, srv813316128=86, srv848347606=88, srv664902431=70, srv843805300=87, srv673575094=72, srv1475941883=16, srv149429439=19, srv1878978779=30, srv1904030274=32, srv985170869=98, srv1554956033=22, srv446448345=56, srv1353540096=10, srv1976453088=38, srv766776048=80, srv1544044432=21, srv1245294538=5, srv676855412=73, srv907432331=94, srv567933417=67, srv1257427698=6, srv1478489484=17, srv1138267611=2, srv1260944144=7, srv1015936688=0, srv992248108=99, srv1392890793=11, srv1599363363=23, srv390138966=54, srv2082007281=44, srv1227366938=4, srv332203072=50, srv1921091157=33, srv1881959713=31, srv1970144413=36, srv1921588734=34, srv1528679002=20, srv753789424=77, srv253642703=46, srv1417594755=12, srv507094004=62, srv1979172918=39, srv478475543=59, srv811994333=85, srv1607715064=24, srv966349708=96, srv1994834283=40, srv2020383804=42, srv623205500=68, srv2003072574=41, srv792859801=82, srv755899798=78, srv681286211=74, srv1029473835=1, srv469169741=58, srv904138546=92, srv764535174=79, srv1768696740=28, srv2092135987=45, srv352840004=51, srv797472118=83, srv545060236=64, srv297862391=47, srv1618408670=25, srv875649862=90, srv895075756=91, srv413357515=55, srv90592646=93, srv681666796=75, srv849649275=89, srv1154535672=3, srv2056396230=43, srv314534318=49, srv528935375=63, srv1938161541=35, srv559313821=66, srv489579947=60, srv64746896=69, srv767751685=81, srv921652753=95, srv495211703=61, srv66856882=71, srv357615321=53, srv353267356=52, srv1657321837=26, srv1464658439=15, srv737391917=76} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-23 08:55:39,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-23 08:55:39,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-23 08:55:39,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-23 08:55:39,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-23 08:55:39,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-23 08:55:39,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-23 08:55:39,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-23 08:55:39,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-23 08:55:39,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-23 08:55:39,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-23 08:55:39,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-23 08:55:39,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-23 08:55:39,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-23 08:55:39,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-23 08:55:39,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-23 08:55:39,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-23 08:55:39,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-23 08:55:39,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-23 08:55:39,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-23 08:55:39,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-23 08:55:39,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-23 08:55:39,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-23 08:55:39,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-23 08:55:39,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-23 08:55:39,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-23 08:55:39,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-23 08:55:39,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-23 08:55:39,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-23 08:55:39,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-23 08:55:39,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-23 08:55:39,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-23 08:55:39,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-23 08:55:39,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-23 08:55:39,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-23 08:55:39,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-23 08:55:39,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-23 08:55:39,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-23 08:55:39,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-23 08:55:39,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-23 08:55:39,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-23 08:55:39,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-23 08:55:39,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-23 08:55:39,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-23 08:55:39,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-23 08:55:39,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-23 08:55:39,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-23 08:55:39,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-23 08:55:39,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-23 08:55:39,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-23 08:55:39,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-23 08:55:39,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-23 08:55:39,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-23 08:55:39,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-23 08:55:39,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-23 08:55:39,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-23 08:55:39,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-23 08:55:39,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-23 08:55:39,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-23 08:55:39,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-23 08:55:39,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-23 08:55:39,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-23 08:55:39,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-23 08:55:39,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-23 08:55:39,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-23 08:55:39,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-23 08:55:39,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-23 08:55:39,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-23 08:55:39,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-23 08:55:39,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-23 08:55:39,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-23 08:55:39,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-23 08:55:39,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-23 08:55:39,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-23 08:55:39,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-23 08:55:39,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-23 08:55:39,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-23 08:55:39,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-23 08:55:39,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-23 08:55:39,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-23 08:55:39,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-23 08:55:39,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-23 08:55:39,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-23 08:55:39,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-23 08:55:39,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-23 08:55:39,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-23 08:55:39,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-23 08:55:39,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-23 08:55:39,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-23 08:55:39,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-23 08:55:39,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-23 08:55:39,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-23 08:55:39,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-23 08:55:39,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-23 08:55:39,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-23 08:55:39,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-23 08:55:39,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-23 08:55:39,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-23 08:55:39,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-23 08:55:39,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-23 08:55:39,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-23 08:55:39,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-23 08:55:39,671 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04062456653197621, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0537546490041772, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04062456653197621, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.16575, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2385, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-23 08:55:39,671 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-23 08:55:39,671 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.1711300401306904, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04062456653197621, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0537546490041772, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04062456653197621, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.16575, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2385, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-23 08:55:40,054 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 562 ms to try 5000 different iterations.  Found a solution that moves 2618 regions; Going from a computed imbalance of 0.1711300401306904 to a new imbalance of 0.015522695531269915. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022161533290822284); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03397695132144565, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29088888888888886, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022161533290822284); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012416666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04516666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-23 08:55:40,073 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1015936688:76 , srv469169741:79 , srv1260944144:80 , srv1445798510:80 , srv800695755:80 , srv992248108:80 , srv1478489484:81 , srv681286211:81 , srv811994333:81 , srv895075756:81 , srv1475941883:82 , srv676855412:82 , srv1029473835:83 , srv1607715064:83 , srv297862391:83 , srv332203072:83 , srv849649275:83 , srv90592646:83 , srv1428901854:84 , srv1482708482:84 , srv1618408670:84 , srv1693000573:84 , srv1768696740:84 , srv1878978779:84 , srv1881959713:84 , srv2092135987:84 , srv353267356:84 , srv489579947:84 , srv681666796:84 , srv848347606:84 , srv1227366938:85 , srv1306952321:85 , srv1921588734:85 , srv1938161541:85 , srv2003072574:85 , srv2056396230:85 , srv2082007281:85 , srv253642703:85 , srv30410366:85 , srv507094004:85 , srv673575094:85 , srv753789424:85 , srv904138546:85 , srv985170869:85 , srv1138267611:86 , srv1245294538:86 , srv1417594755:86 , srv1464658439:86 , srv149429439:86 , srv1554956033:86 , srv357615321:86 , srv495211703:86 , srv557445629:86 , srv623205500:86 , srv66856882:86 , srv907432331:86 , srv921652753:86 , srv976077466:86 , srv1333830018:87 , srv1353540096:87 , srv1528679002:87 , srv1599363363:87 , srv1788563628:87 , srv1921091157:87 , srv1994834283:87 , srv352840004:87 , srv390138966:87 , srv413357515:87 , srv528935375:87 , srv545060236:87 , srv559313821:87 , srv664902431:87 , srv764535174:87 , srv767751685:87 , srv1544044432:88 , srv1657321837:88 , srv1970144413:88 , srv1975499969:88 , srv1976453088:88 , srv1979172918:88 , srv2020383804:88 , srv453906192:88 , srv478475543:88 , srv1257427698:89 , srv314534318:89 , srv567933417:89 , srv64746896:89 , srv737391917:89 , srv755899798:89 , srv766776048:89 , srv792859801:89 , srv797472118:89 , srv813316128:89 , srv1154535672:90 , srv446448345:90 , srv843805300:90 , srv966349708:90 , srv875649862:91 , srv1392890793:92 , srv1904030274:520 }
2023-04-23 08:55:40,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-23 08:55:40,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv453906192=57, srv1482708482=18, srv1445798510=14, srv1428901854=13, srv1693000573=27, srv30410366=48, srv976077466=97, srv1975499969=37, srv557445629=65, srv1306952321=8, srv1333830018=9, srv800695755=84, srv1788563628=29, srv813316128=86, srv848347606=88, srv664902431=70, srv843805300=87, srv673575094=72, srv1475941883=16, srv149429439=19, srv1878978779=30, srv1904030274=32, srv985170869=98, srv1554956033=22, srv446448345=56, srv1353540096=10, srv1976453088=38, srv766776048=80, srv1544044432=21, srv1245294538=5, srv676855412=73, srv907432331=94, srv567933417=67, srv1257427698=6, srv1478489484=17, srv1138267611=2, srv1260944144=7, srv1015936688=0, srv992248108=99, srv1392890793=11, srv1599363363=23, srv390138966=54, srv2082007281=44, srv1227366938=4, srv332203072=50, srv1921091157=33, srv1881959713=31, srv1970144413=36, srv1921588734=34, srv1528679002=20, srv753789424=77, srv253642703=46, srv1417594755=12, srv507094004=62, srv1979172918=39, srv478475543=59, srv811994333=85, srv1607715064=24, srv966349708=96, srv1994834283=40, srv2020383804=42, srv623205500=68, srv2003072574=41, srv792859801=82, srv755899798=78, srv681286211=74, srv1029473835=1, srv469169741=58, srv904138546=92, srv764535174=79, srv1768696740=28, srv2092135987=45, srv352840004=51, srv797472118=83, srv545060236=64, srv297862391=47, srv1618408670=25, srv875649862=90, srv895075756=91, srv413357515=55, srv90592646=93, srv681666796=75, srv849649275=89, srv1154535672=3, srv2056396230=43, srv314534318=49, srv528935375=63, srv1938161541=35, srv559313821=66, srv489579947=60, srv64746896=69, srv767751685=81, srv921652753=95, srv495211703=61, srv66856882=71, srv357615321=53, srv353267356=52, srv1657321837=26, srv1464658439=15, srv737391917=76} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-23 08:55:40,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-23 08:55:40,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-23 08:55:40,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-23 08:55:40,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-23 08:55:40,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-23 08:55:40,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-23 08:55:40,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-23 08:55:40,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-23 08:55:40,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-23 08:55:40,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-23 08:55:40,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-23 08:55:40,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-23 08:55:40,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-23 08:55:40,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-23 08:55:40,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-23 08:55:40,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-23 08:55:40,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-23 08:55:40,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-23 08:55:40,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-23 08:55:40,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-23 08:55:40,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-23 08:55:40,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-23 08:55:40,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-23 08:55:40,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-23 08:55:40,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-23 08:55:40,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-23 08:55:40,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-23 08:55:40,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-23 08:55:40,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-23 08:55:40,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-23 08:55:40,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-23 08:55:40,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-23 08:55:40,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-23 08:55:40,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-23 08:55:40,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-23 08:55:40,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-23 08:55:40,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-23 08:55:40,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-23 08:55:40,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-23 08:55:40,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-23 08:55:40,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-23 08:55:40,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-23 08:55:40,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-23 08:55:40,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-23 08:55:40,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-23 08:55:40,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-23 08:55:40,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-23 08:55:40,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-23 08:55:40,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-23 08:55:40,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-23 08:55:40,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-23 08:55:40,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-23 08:55:40,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-23 08:55:40,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-23 08:55:40,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-23 08:55:40,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-23 08:55:40,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-23 08:55:40,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-23 08:55:40,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-23 08:55:40,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-23 08:55:40,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-23 08:55:40,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-23 08:55:40,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-23 08:55:40,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-23 08:55:40,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-23 08:55:40,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-23 08:55:40,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-23 08:55:40,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-23 08:55:40,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-23 08:55:40,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-23 08:55:40,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-23 08:55:40,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-23 08:55:40,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-23 08:55:40,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-23 08:55:40,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-23 08:55:40,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-23 08:55:40,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-23 08:55:40,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-23 08:55:40,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-23 08:55:40,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-23 08:55:40,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-23 08:55:40,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-23 08:55:40,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-23 08:55:40,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-23 08:55:40,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-23 08:55:40,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-23 08:55:40,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-23 08:55:40,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-23 08:55:40,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-23 08:55:40,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-23 08:55:40,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-23 08:55:40,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-23 08:55:40,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-23 08:55:40,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-23 08:55:40,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-23 08:55:40,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-23 08:55:40,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-23 08:55:40,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-23 08:55:40,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-23 08:55:40,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-23 08:55:40,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-23 08:55:40,291 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022161533290822284); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03397695132144565, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022161533290822284); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012416666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04516666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-23 08:55:40,291 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-23 08:55:40,291 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.015504363903460177, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022161533290822284); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03397695132144565, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022161533290822284); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012416666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04516666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-23 08:55:40,634 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 472 ms to try 5000 different iterations.  Found a solution that moves 980 regions; Going from a computed imbalance of 0.015504363903460177 to a new imbalance of 9.684230309205042E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006201504711026421); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008645662192537639); MoveCostFunction : (multiplier=7.0, imbalance=0.10888888888888888, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006201504711026421); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.833333333333334E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004083333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-23 08:55:40,636 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1392890793:80 , srv1306952321:82 , srv1417594755:82 , srv875649862:82 , srv1693000573:83 , srv966349708:83 , srv1015936688:84 , srv1921091157:84 , srv1938161541:84 , srv1970144413:84 , srv30410366:84 , srv314534318:84 , srv559313821:84 , srv66856882:84 , srv753789424:84 , srv755899798:84 , srv797472118:84 , srv843805300:84 , srv848347606:84 , srv1138267611:85 , srv1154535672:85 , srv1260944144:85 , srv1353540096:85 , srv1428901854:85 , srv1528679002:85 , srv1599363363:85 , srv1878978779:85 , srv1976453088:85 , srv1979172918:85 , srv1994834283:85 , srv2020383804:85 , srv2056396230:85 , srv2082007281:85 , srv353267356:85 , srv446448345:85 , srv489579947:85 , srv545060236:85 , srv557445629:85 , srv567933417:85 , srv623205500:85 , srv64746896:85 , srv664902431:85 , srv681286211:85 , srv764535174:85 , srv766776048:85 , srv767751685:85 , srv811994333:85 , srv813316128:85 , srv895075756:85 , srv921652753:85 , srv976077466:85 , srv1227366938:86 , srv1245294538:86 , srv1257427698:86 , srv1464658439:86 , srv1475941883:86 , srv1478489484:86 , srv1482708482:86 , srv1544044432:86 , srv1618408670:86 , srv1657321837:86 , srv1975499969:86 , srv352840004:86 , srv357615321:86 , srv390138966:86 , srv413357515:86 , srv478475543:86 , srv495211703:86 , srv507094004:86 , srv528935375:86 , srv673575094:86 , srv681666796:86 , srv792859801:86 , srv904138546:86 , srv907432331:86 , srv985170869:86 , srv992248108:86 , srv1333830018:87 , srv1445798510:87 , srv149429439:87 , srv1554956033:87 , srv1607715064:87 , srv1768696740:87 , srv1788563628:87 , srv1881959713:87 , srv1921588734:87 , srv2003072574:87 , srv2092135987:87 , srv253642703:87 , srv297862391:87 , srv453906192:87 , srv469169741:87 , srv676855412:87 , srv737391917:87 , srv800695755:87 , srv90592646:87 , srv332203072:88 , srv849649275:88 , srv1029473835:89 , srv1904030274:542 }
2023-04-23 08:55:40,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-23 08:55:40,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv453906192=57, srv1482708482=18, srv1445798510=14, srv1428901854=13, srv1693000573=27, srv30410366=48, srv976077466=97, srv1975499969=37, srv557445629=65, srv1306952321=8, srv1333830018=9, srv800695755=84, srv1788563628=29, srv813316128=86, srv848347606=88, srv664902431=70, srv843805300=87, srv673575094=72, srv1475941883=16, srv149429439=19, srv1878978779=30, srv1904030274=32, srv985170869=98, srv1554956033=22, srv446448345=56, srv1353540096=10, srv1976453088=38, srv766776048=80, srv1544044432=21, srv1245294538=5, srv676855412=73, srv907432331=94, srv567933417=67, srv1257427698=6, srv1478489484=17, srv1138267611=2, srv1260944144=7, srv1015936688=0, srv992248108=99, srv1392890793=11, srv1599363363=23, srv390138966=54, srv2082007281=44, srv1227366938=4, srv332203072=50, srv1921091157=33, srv1881959713=31, srv1970144413=36, srv1921588734=34, srv1528679002=20, srv753789424=77, srv253642703=46, srv1417594755=12, srv507094004=62, srv1979172918=39, srv478475543=59, srv811994333=85, srv1607715064=24, srv966349708=96, srv1994834283=40, srv2020383804=42, srv623205500=68, srv2003072574=41, srv792859801=82, srv755899798=78, srv681286211=74, srv1029473835=1, srv469169741=58, srv904138546=92, srv764535174=79, srv1768696740=28, srv2092135987=45, srv352840004=51, srv797472118=83, srv545060236=64, srv297862391=47, srv1618408670=25, srv875649862=90, srv895075756=91, srv413357515=55, srv90592646=93, srv681666796=75, srv849649275=89, srv1154535672=3, srv2056396230=43, srv314534318=49, srv528935375=63, srv1938161541=35, srv559313821=66, srv489579947=60, srv64746896=69, srv767751685=81, srv921652753=95, srv495211703=61, srv66856882=71, srv357615321=53, srv353267356=52, srv1657321837=26, srv1464658439=15, srv737391917=76} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-23 08:55:40,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-23 08:55:40,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-23 08:55:40,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-23 08:55:40,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-23 08:55:40,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-23 08:55:40,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-23 08:55:40,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-23 08:55:40,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-23 08:55:40,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-23 08:55:40,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-23 08:55:40,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-23 08:55:40,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-23 08:55:40,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-23 08:55:40,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-23 08:55:40,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-23 08:55:40,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-23 08:55:40,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-23 08:55:40,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-23 08:55:40,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-23 08:55:40,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-23 08:55:40,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-23 08:55:40,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-23 08:55:40,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-23 08:55:40,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-23 08:55:40,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-23 08:55:40,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-23 08:55:40,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-23 08:55:40,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-23 08:55:40,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-23 08:55:40,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-23 08:55:40,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-23 08:55:40,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-23 08:55:40,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-23 08:55:40,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-23 08:55:40,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-23 08:55:40,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-23 08:55:40,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-23 08:55:40,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-23 08:55:40,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-23 08:55:40,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-23 08:55:40,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-23 08:55:40,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-23 08:55:40,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-23 08:55:40,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-23 08:55:40,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-23 08:55:40,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-23 08:55:40,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-23 08:55:40,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-23 08:55:40,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-23 08:55:40,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-23 08:55:40,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-23 08:55:40,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-23 08:55:40,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-23 08:55:40,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-23 08:55:40,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-23 08:55:40,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-23 08:55:40,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-23 08:55:40,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-23 08:55:40,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-23 08:55:40,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-23 08:55:40,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-23 08:55:40,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-23 08:55:40,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-23 08:55:40,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-23 08:55:40,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-23 08:55:40,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-23 08:55:40,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-23 08:55:40,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-23 08:55:40,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-23 08:55:40,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-23 08:55:40,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-23 08:55:40,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-23 08:55:40,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-23 08:55:40,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-23 08:55:40,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-23 08:55:40,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-23 08:55:40,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-23 08:55:40,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-23 08:55:40,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-23 08:55:40,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-23 08:55:40,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-23 08:55:40,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-23 08:55:40,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-23 08:55:40,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-23 08:55:40,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-23 08:55:40,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-23 08:55:40,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-23 08:55:40,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-23 08:55:40,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-23 08:55:40,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-23 08:55:40,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-23 08:55:40,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-23 08:55:40,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-23 08:55:40,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-23 08:55:40,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-23 08:55:40,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-23 08:55:40,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-23 08:55:40,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-23 08:55:40,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-23 08:55:40,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-23 08:55:40,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-23 08:55:40,808 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006201504711026421); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008645662192537639); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006201504711026421); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.833333333333334E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004083333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-23 08:55:40,808 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-23 08:55:40,808 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=9.615609242537575E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006201504711026421); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008645662192537639); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006201504711026421); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.833333333333334E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004083333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-23 08:55:41,172 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 481 ms to try 5000 different iterations.  Found a solution that moves 261 regions; Going from a computed imbalance of 9.615609242537575E-4 to a new imbalance of 3.4728321167548824E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.868392354571063E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.029, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.868392354571063E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-23 08:55:41,173 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1015936688:84 , srv1029473835:84 , srv1154535672:84 , srv1257427698:84 , srv1306952321:84 , srv1353540096:84 , srv1445798510:84 , srv1464658439:84 , srv1475941883:84 , srv1478489484:84 , srv1482708482:84 , srv149429439:84 , srv1554956033:84 , srv1607715064:84 , srv1618408670:84 , srv1693000573:84 , srv1788563628:84 , srv1878978779:84 , srv1881959713:84 , srv1921588734:84 , srv1975499969:84 , srv1976453088:84 , srv1994834283:84 , srv2020383804:84 , srv253642703:84 , srv297862391:84 , srv314534318:84 , srv332203072:84 , srv446448345:84 , srv469169741:84 , srv478475543:84 , srv495211703:84 , srv545060236:84 , srv557445629:84 , srv64746896:84 , srv664902431:84 , srv673575094:84 , srv681286211:84 , srv737391917:84 , srv766776048:84 , srv792859801:84 , srv797472118:84 , srv813316128:84 , srv849649275:84 , srv904138546:84 , srv90592646:84 , srv985170869:84 , srv1138267611:85 , srv1227366938:85 , srv1245294538:85 , srv1260944144:85 , srv1333830018:85 , srv1417594755:85 , srv1428901854:85 , srv1528679002:85 , srv1544044432:85 , srv1599363363:85 , srv1657321837:85 , srv1768696740:85 , srv1921091157:85 , srv1938161541:85 , srv1970144413:85 , srv1979172918:85 , srv2003072574:85 , srv2056396230:85 , srv2082007281:85 , srv2092135987:85 , srv30410366:85 , srv352840004:85 , srv353267356:85 , srv357615321:85 , srv413357515:85 , srv453906192:85 , srv489579947:85 , srv507094004:85 , srv528935375:85 , srv567933417:85 , srv623205500:85 , srv66856882:85 , srv676855412:85 , srv681666796:85 , srv753789424:85 , srv755899798:85 , srv764535174:85 , srv767751685:85 , srv800695755:85 , srv811994333:85 , srv843805300:85 , srv848347606:85 , srv875649862:85 , srv895075756:85 , srv907432331:85 , srv921652753:85 , srv966349708:85 , srv976077466:85 , srv992248108:85 , srv1392890793:86 , srv390138966:86 , srv559313821:86 , srv1904030274:629 }
2023-04-23 08:55:41,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-23 08:55:41,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv453906192=57, srv1482708482=18, srv1445798510=14, srv1428901854=13, srv1693000573=27, srv30410366=48, srv976077466=97, srv1975499969=37, srv557445629=65, srv1306952321=8, srv1333830018=9, srv800695755=84, srv1788563628=29, srv813316128=86, srv848347606=88, srv664902431=70, srv843805300=87, srv673575094=72, srv1475941883=16, srv149429439=19, srv1878978779=30, srv1904030274=32, srv985170869=98, srv1554956033=22, srv446448345=56, srv1353540096=10, srv1976453088=38, srv766776048=80, srv1544044432=21, srv1245294538=5, srv676855412=73, srv907432331=94, srv567933417=67, srv1257427698=6, srv1478489484=17, srv1138267611=2, srv1260944144=7, srv1015936688=0, srv992248108=99, srv1392890793=11, srv1599363363=23, srv390138966=54, srv2082007281=44, srv1227366938=4, srv332203072=50, srv1921091157=33, srv1881959713=31, srv1970144413=36, srv1921588734=34, srv1528679002=20, srv753789424=77, srv253642703=46, srv1417594755=12, srv507094004=62, srv1979172918=39, srv478475543=59, srv811994333=85, srv1607715064=24, srv966349708=96, srv1994834283=40, srv2020383804=42, srv623205500=68, srv2003072574=41, srv792859801=82, srv755899798=78, srv681286211=74, srv1029473835=1, srv469169741=58, srv904138546=92, srv764535174=79, srv1768696740=28, srv2092135987=45, srv352840004=51, srv797472118=83, srv545060236=64, srv297862391=47, srv1618408670=25, srv875649862=90, srv895075756=91, srv413357515=55, srv90592646=93, srv681666796=75, srv849649275=89, srv1154535672=3, srv2056396230=43, srv314534318=49, srv528935375=63, srv1938161541=35, srv559313821=66, srv489579947=60, srv64746896=69, srv767751685=81, srv921652753=95, srv495211703=61, srv66856882=71, srv357615321=53, srv353267356=52, srv1657321837=26, srv1464658439=15, srv737391917=76} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-23 08:55:41,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-23 08:55:41,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-23 08:55:41,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-23 08:55:41,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-23 08:55:41,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-23 08:55:41,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-23 08:55:41,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-23 08:55:41,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-23 08:55:41,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-23 08:55:41,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-23 08:55:41,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-23 08:55:41,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-23 08:55:41,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-23 08:55:41,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-23 08:55:41,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-23 08:55:41,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-23 08:55:41,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-23 08:55:41,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-23 08:55:41,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-23 08:55:41,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-23 08:55:41,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-23 08:55:41,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-23 08:55:41,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-23 08:55:41,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-23 08:55:41,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-23 08:55:41,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-23 08:55:41,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-23 08:55:41,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-23 08:55:41,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-23 08:55:41,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-23 08:55:41,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-23 08:55:41,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-23 08:55:41,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-23 08:55:41,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-23 08:55:41,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-23 08:55:41,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-23 08:55:41,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-23 08:55:41,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-23 08:55:41,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-23 08:55:41,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-23 08:55:41,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-23 08:55:41,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-23 08:55:41,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-23 08:55:41,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-23 08:55:41,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-23 08:55:41,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-23 08:55:41,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-23 08:55:41,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-23 08:55:41,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-23 08:55:41,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-23 08:55:41,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-23 08:55:41,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-23 08:55:41,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-23 08:55:41,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-23 08:55:41,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-23 08:55:41,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-23 08:55:41,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-23 08:55:41,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-23 08:55:41,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-23 08:55:41,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-23 08:55:41,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-23 08:55:41,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-23 08:55:41,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-23 08:55:41,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-23 08:55:41,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-23 08:55:41,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-23 08:55:41,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-23 08:55:41,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-23 08:55:41,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-23 08:55:41,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-23 08:55:41,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-23 08:55:41,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-23 08:55:41,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-23 08:55:41,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-23 08:55:41,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-23 08:55:41,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-23 08:55:41,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-23 08:55:41,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-23 08:55:41,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-23 08:55:41,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-23 08:55:41,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-23 08:55:41,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-23 08:55:41,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-23 08:55:41,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-23 08:55:41,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-23 08:55:41,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-23 08:55:41,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-23 08:55:41,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-23 08:55:41,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-23 08:55:41,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-23 08:55:41,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-23 08:55:41,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-23 08:55:41,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-23 08:55:41,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-23 08:55:41,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-23 08:55:41,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-23 08:55:41,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-23 08:55:41,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-23 08:55:41,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-23 08:55:41,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-23 08:55:41,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-23 08:55:41,354 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.868392354571063E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.868392354571063E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-23 08:55:41,354 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-23 08:55:41,355 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=3.290076010630302E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.868392354571063E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.868392354571063E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-23 08:55:41,644 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 424 ms to try 5000 different iterations.  Found a solution that moves 24 regions; Going from a computed imbalance of 3.290076010630302E-5 to a new imbalance of 7.857473392189477E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017082358106351274); MoveCostFunction : (multiplier=7.0, imbalance=0.0026666666666666666); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-23 08:55:41,644 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1392890793:83 , srv2003072574:83 , srv528935375:83 , srv567933417:83 , srv767751685:83 , srv875649862:83 , srv1015936688:84 , srv1029473835:84 , srv1138267611:84 , srv1154535672:84 , srv1227366938:84 , srv1245294538:84 , srv1257427698:84 , srv1260944144:84 , srv1333830018:84 , srv1353540096:84 , srv1417594755:84 , srv1428901854:84 , srv1464658439:84 , srv1478489484:84 , srv1482708482:84 , srv149429439:84 , srv1528679002:84 , srv1544044432:84 , srv1554956033:84 , srv1599363363:84 , srv1607715064:84 , srv1618408670:84 , srv1657321837:84 , srv1693000573:84 , srv1768696740:84 , srv1788563628:84 , srv1878978779:84 , srv1881959713:84 , srv1921091157:84 , srv1921588734:84 , srv1938161541:84 , srv1970144413:84 , srv1975499969:84 , srv1976453088:84 , srv1979172918:84 , srv1994834283:84 , srv2056396230:84 , srv2082007281:84 , srv2092135987:84 , srv253642703:84 , srv297862391:84 , srv30410366:84 , srv314534318:84 , srv332203072:84 , srv352840004:84 , srv353267356:84 , srv357615321:84 , srv390138966:84 , srv413357515:84 , srv453906192:84 , srv469169741:84 , srv478475543:84 , srv489579947:84 , srv495211703:84 , srv507094004:84 , srv545060236:84 , srv557445629:84 , srv559313821:84 , srv623205500:84 , srv64746896:84 , srv664902431:84 , srv66856882:84 , srv673575094:84 , srv676855412:84 , srv681286211:84 , srv681666796:84 , srv753789424:84 , srv755899798:84 , srv764535174:84 , srv766776048:84 , srv792859801:84 , srv797472118:84 , srv800695755:84 , srv811994333:84 , srv813316128:84 , srv843805300:84 , srv848347606:84 , srv849649275:84 , srv895075756:84 , srv904138546:84 , srv90592646:84 , srv907432331:84 , srv921652753:84 , srv966349708:84 , srv976077466:84 , srv985170869:84 , srv992248108:84 , srv1306952321:85 , srv1445798510:85 , srv1475941883:85 , srv2020383804:85 , srv446448345:85 , srv737391917:85 , srv1904030274:684 }
2023-04-23 08:55:41,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-23 08:55:41,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv453906192=57, srv1482708482=18, srv1445798510=14, srv1428901854=13, srv1693000573=27, srv30410366=48, srv976077466=97, srv1975499969=37, srv557445629=65, srv1306952321=8, srv1333830018=9, srv800695755=84, srv1788563628=29, srv813316128=86, srv848347606=88, srv664902431=70, srv843805300=87, srv673575094=72, srv1475941883=16, srv149429439=19, srv1878978779=30, srv1904030274=32, srv985170869=98, srv1554956033=22, srv446448345=56, srv1353540096=10, srv1976453088=38, srv766776048=80, srv1544044432=21, srv1245294538=5, srv676855412=73, srv907432331=94, srv567933417=67, srv1257427698=6, srv1478489484=17, srv1138267611=2, srv1260944144=7, srv1015936688=0, srv992248108=99, srv1392890793=11, srv1599363363=23, srv390138966=54, srv2082007281=44, srv1227366938=4, srv332203072=50, srv1921091157=33, srv1881959713=31, srv1970144413=36, srv1921588734=34, srv1528679002=20, srv753789424=77, srv253642703=46, srv1417594755=12, srv507094004=62, srv1979172918=39, srv478475543=59, srv811994333=85, srv1607715064=24, srv966349708=96, srv1994834283=40, srv2020383804=42, srv623205500=68, srv2003072574=41, srv792859801=82, srv755899798=78, srv681286211=74, srv1029473835=1, srv469169741=58, srv904138546=92, srv764535174=79, srv1768696740=28, srv2092135987=45, srv352840004=51, srv797472118=83, srv545060236=64, srv297862391=47, srv1618408670=25, srv875649862=90, srv895075756=91, srv413357515=55, srv90592646=93, srv681666796=75, srv849649275=89, srv1154535672=3, srv2056396230=43, srv314534318=49, srv528935375=63, srv1938161541=35, srv559313821=66, srv489579947=60, srv64746896=69, srv767751685=81, srv921652753=95, srv495211703=61, srv66856882=71, srv357615321=53, srv353267356=52, srv1657321837=26, srv1464658439=15, srv737391917=76} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-23 08:55:41,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-23 08:55:41,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-23 08:55:41,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-23 08:55:41,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-23 08:55:41,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-23 08:55:41,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-23 08:55:41,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-23 08:55:41,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-23 08:55:41,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-23 08:55:41,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-23 08:55:41,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-23 08:55:41,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-23 08:55:41,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-23 08:55:41,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-23 08:55:41,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-23 08:55:41,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-23 08:55:41,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-23 08:55:41,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-23 08:55:41,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-23 08:55:41,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-23 08:55:41,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-23 08:55:41,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-23 08:55:41,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-23 08:55:41,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-23 08:55:41,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-23 08:55:41,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-23 08:55:41,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-23 08:55:41,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-23 08:55:41,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-23 08:55:41,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-23 08:55:41,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-23 08:55:41,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-23 08:55:41,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-23 08:55:41,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-23 08:55:41,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-23 08:55:41,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-23 08:55:41,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-23 08:55:41,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-23 08:55:41,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-23 08:55:41,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-23 08:55:41,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-23 08:55:41,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-23 08:55:41,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-23 08:55:41,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-23 08:55:41,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-23 08:55:41,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-23 08:55:41,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-23 08:55:41,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-23 08:55:41,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-23 08:55:41,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-23 08:55:41,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-23 08:55:41,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-23 08:55:41,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-23 08:55:41,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-23 08:55:41,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-23 08:55:41,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-23 08:55:41,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-23 08:55:41,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-23 08:55:41,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-23 08:55:41,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-23 08:55:41,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-23 08:55:41,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-23 08:55:41,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-23 08:55:41,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-23 08:55:41,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-23 08:55:41,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-23 08:55:41,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-23 08:55:41,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-23 08:55:41,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-23 08:55:41,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-23 08:55:41,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-23 08:55:41,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-23 08:55:41,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-23 08:55:41,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-23 08:55:41,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-23 08:55:41,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-23 08:55:41,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-23 08:55:41,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-23 08:55:41,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-23 08:55:41,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-23 08:55:41,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-23 08:55:41,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-23 08:55:41,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-23 08:55:41,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-23 08:55:41,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-23 08:55:41,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-23 08:55:41,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-23 08:55:41,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-23 08:55:41,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-23 08:55:41,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-23 08:55:41,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-23 08:55:41,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-23 08:55:41,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-23 08:55:41,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-23 08:55:41,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-23 08:55:41,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-23 08:55:41,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-23 08:55:41,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-23 08:55:41,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-23 08:55:41,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-23 08:55:41,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-23 08:55:41,827 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=7.689421800350782E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017082358106351274); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-23 08:55:41,827 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1392890793:83 , srv2003072574:83 , srv528935375:83 , srv567933417:83 , srv767751685:83 , srv875649862:83 , srv1015936688:84 , srv1029473835:84 , srv1138267611:84 , srv1154535672:84 , srv1227366938:84 , srv1245294538:84 , srv1257427698:84 , srv1260944144:84 , srv1333830018:84 , srv1353540096:84 , srv1417594755:84 , srv1428901854:84 , srv1464658439:84 , srv1478489484:84 , srv1482708482:84 , srv149429439:84 , srv1528679002:84 , srv1544044432:84 , srv1554956033:84 , srv1599363363:84 , srv1607715064:84 , srv1618408670:84 , srv1657321837:84 , srv1693000573:84 , srv1768696740:84 , srv1788563628:84 , srv1878978779:84 , srv1881959713:84 , srv1921091157:84 , srv1921588734:84 , srv1938161541:84 , srv1970144413:84 , srv1975499969:84 , srv1976453088:84 , srv1979172918:84 , srv1994834283:84 , srv2056396230:84 , srv2082007281:84 , srv2092135987:84 , srv253642703:84 , srv297862391:84 , srv30410366:84 , srv314534318:84 , srv332203072:84 , srv352840004:84 , srv353267356:84 , srv357615321:84 , srv390138966:84 , srv413357515:84 , srv453906192:84 , srv469169741:84 , srv478475543:84 , srv489579947:84 , srv495211703:84 , srv507094004:84 , srv545060236:84 , srv557445629:84 , srv559313821:84 , srv623205500:84 , srv64746896:84 , srv664902431:84 , srv66856882:84 , srv673575094:84 , srv676855412:84 , srv681286211:84 , srv681666796:84 , srv753789424:84 , srv755899798:84 , srv764535174:84 , srv766776048:84 , srv792859801:84 , srv797472118:84 , srv800695755:84 , srv811994333:84 , srv813316128:84 , srv843805300:84 , srv848347606:84 , srv849649275:84 , srv895075756:84 , srv904138546:84 , srv90592646:84 , srv907432331:84 , srv921652753:84 , srv966349708:84 , srv976077466:84 , srv985170869:84 , srv992248108:84 , srv1306952321:85 , srv1445798510:85 , srv1475941883:85 , srv2020383804:85 , srv446448345:85 , srv737391917:85 , srv1904030274:684 }
2023-04-23 08:55:41,859 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=491 (was 455) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 169), AvailableMemoryMB=12583 (was 13068)
2023-04-23 08:55:41,871 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=491, ProcessCount=169, AvailableMemoryMB=12583
2023-04-23 08:55:41,872 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-23 08:55:41,872 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-04-23 08:55:41,881 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1118326945:84 , srv1147210236:84 , srv1261298872:84 , srv1338015321:84 , srv1435178070:84 , srv1438539537:84 , srv1457361213:84 , srv1495641187:84 , srv1498082553:84 , srv1508848586:84 , srv1546419316:84 , srv1594375724:84 , srv1759593663:84 , srv177382218:84 , srv1780531039:84 , srv1913228418:84 , srv1944877110:84 , srv2089552160:84 , srv430998403:84 , srv474822504:84 , srv585012598:84 , srv592135650:84 , srv606167246:84 , srv738796781:84 , srv766922774:84 , srv779003046:84 , srv791210266:84 , srv826883096:84 , srv965126947:84 , srv1815007476:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-04-23 08:55:41,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-23 08:55:41,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1118326945=0, srv606167246=23, srv766922774=25, srv2089552160=18, srv1944877110=17, srv474822504=20, srv1498082553=8, srv965126947=29, srv1594375724=11, srv779003046=26, srv1261298872=2, srv592135650=22, srv1780531039=14, srv826883096=28, srv1457361213=6, srv1338015321=3, srv738796781=24, srv177382218=13, srv1147210236=1, srv1546419316=10, srv1508848586=9, srv1815007476=15, srv1495641187=7, srv1435178070=4, srv1913228418=16, srv585012598=21, srv1438539537=5, srv1759593663=12, srv430998403=19, srv791210266=27} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-23 08:55:41,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-23 08:55:41,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-23 08:55:41,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-23 08:55:41,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-23 08:55:41,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-23 08:55:41,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-23 08:55:41,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-23 08:55:41,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-23 08:55:41,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-23 08:55:41,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-23 08:55:41,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-23 08:55:41,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-23 08:55:41,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-23 08:55:41,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-23 08:55:41,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-23 08:55:41,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-23 08:55:41,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-23 08:55:41,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-23 08:55:41,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-23 08:55:41,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-23 08:55:41,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-23 08:55:41,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-23 08:55:41,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-23 08:55:41,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-23 08:55:41,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-23 08:55:41,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-23 08:55:41,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-23 08:55:41,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-23 08:55:41,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-23 08:55:41,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-23 08:55:41,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-04-23 08:55:41,921 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.2037089153262799, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-23 08:55:41,922 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912834923404762, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.2037089153262799, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-04-23 08:57:41,901 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6181915 different iterations.  Found a solution that moves 1857 regions; Going from a computed imbalance of 0.9912834923404762 to a new imbalance of 4.334330639506329E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6877777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-23 08:57:41,906 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1118326945:90 , srv1147210236:90 , srv1261298872:90 , srv1338015321:90 , srv1435178070:90 , srv1438539537:90 , srv1457361213:90 , srv1495641187:90 , srv1498082553:90 , srv1508848586:90 , srv1546419316:90 , srv1594375724:90 , srv1759593663:90 , srv177382218:90 , srv1780531039:90 , srv1815007476:90 , srv1913228418:90 , srv1944877110:90 , srv2089552160:90 , srv430998403:90 , srv474822504:90 , srv585012598:90 , srv592135650:90 , srv606167246:90 , srv738796781:90 , srv766922774:90 , srv779003046:90 , srv791210266:90 , srv826883096:90 , srv965126947:90 }
2023-04-23 08:57:41,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-23 08:57:41,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1118326945=0, srv606167246=23, srv766922774=25, srv2089552160=18, srv1944877110=17, srv474822504=20, srv1498082553=8, srv965126947=29, srv1594375724=11, srv779003046=26, srv1261298872=2, srv592135650=22, srv1780531039=14, srv826883096=28, srv1457361213=6, srv1338015321=3, srv738796781=24, srv177382218=13, srv1147210236=1, srv1546419316=10, srv1508848586=9, srv1815007476=15, srv1495641187=7, srv1435178070=4, srv1913228418=16, srv585012598=21, srv1438539537=5, srv1759593663=12, srv430998403=19, srv791210266=27} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-23 08:57:41,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-23 08:57:41,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-23 08:57:41,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-23 08:57:41,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-23 08:57:41,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-23 08:57:41,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-23 08:57:41,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-23 08:57:41,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-23 08:57:41,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-23 08:57:41,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-23 08:57:41,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-23 08:57:41,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-23 08:57:41,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-23 08:57:41,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-23 08:57:41,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-23 08:57:41,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-23 08:57:41,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-23 08:57:41,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-23 08:57:41,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-23 08:57:41,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-23 08:57:41,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-23 08:57:41,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-23 08:57:41,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-23 08:57:41,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-23 08:57:41,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-23 08:57:41,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-23 08:57:41,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-23 08:57:41,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-23 08:57:41,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-23 08:57:41,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-23 08:57:41,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-04-23 08:57:41,924 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-23 08:57:41,924 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1118326945:90 , srv1147210236:90 , srv1261298872:90 , srv1338015321:90 , srv1435178070:90 , srv1438539537:90 , srv1457361213:90 , srv1495641187:90 , srv1498082553:90 , srv1508848586:90 , srv1546419316:90 , srv1594375724:90 , srv1759593663:90 , srv177382218:90 , srv1780531039:90 , srv1815007476:90 , srv1913228418:90 , srv1944877110:90 , srv2089552160:90 , srv430998403:90 , srv474822504:90 , srv585012598:90 , srv592135650:90 , srv606167246:90 , srv738796781:90 , srv766922774:90 , srv779003046:90 , srv791210266:90 , srv826883096:90 , srv965126947:90 }
2023-04-23 08:57:41,937 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=269 (was 491), ProcessCount=169 (was 169), AvailableMemoryMB=12667 (was 12583) - AvailableMemoryMB LEAK? -