2023-02-14 15:54:01,669 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-02-14 15:54:02,003 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-02-14 15:54:02,108 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-02-14 15:54:02,166 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-14 15:54:02,173 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-02-14 15:54:02,186 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=338, ProcessCount=168, AvailableMemoryMB=5305
2023-02-14 15:54:02,188 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-14 15:54:02,188 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-02-14 15:54:02,323 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1011619801:84 , srv1042779744:84 , srv1065429010:84 , srv1080582355:84 , srv1090186448:84 , srv1102953625:84 , srv1105557363:84 , srv1111392893:84 , srv1119858410:84 , srv120108661:84 , srv1208153922:84 , srv1228054314:84 , srv1231562670:84 , srv1240701023:84 , srv1278446407:84 , srv1303404149:84 , srv1322968277:84 , srv1401764352:84 , srv1406827548:84 , srv1421930010:84 , srv1426863158:84 , srv1428405325:84 , srv1471042565:84 , srv1474695358:84 , srv1561762135:84 , srv1589920785:84 , srv1590029266:84 , srv1591564338:84 , srv1595737586:84 , srv1616566360:84 , srv1625464073:84 , srv1628963987:84 , srv1684709033:84 , srv1727079872:84 , srv1748177190:84 , srv1810167936:84 , srv1833824735:84 , srv1838835049:84 , srv1845986435:84 , srv1853791311:84 , srv1855828918:84 , srv1866207076:84 , srv1871762428:84 , srv1881157965:84 , srv1925493852:84 , srv1973594251:84 , srv1975810639:84 , srv1978469510:84 , srv2002833269:84 , srv2032489014:84 , srv2039760844:84 , srv2043710935:84 , srv2046795662:84 , srv2049784388:84 , srv2050834476:84 , srv2064454636:84 , srv2082442318:84 , srv2096314520:84 , srv2099352059:84 , srv2127430764:84 , srv214433328:84 , srv243760619:84 , srv253964902:84 , srv294732698:84 , srv331322030:84 , srv355229554:84 , srv376453246:84 , srv377428337:84 , srv38115448:84 , srv414213644:84 , srv419383482:84 , srv444527564:84 , srv465354439:84 , srv469145417:84 , srv472414025:84 , srv501357554:84 , srv522457048:84 , srv539318666:84 , srv592346347:84 , srv593072564:84 , srv621699242:84 , srv673019616:84 , srv723778758:84 , srv750743354:84 , srv781580449:84 , srv785693732:84 , srv802500994:84 , srv81977692:84 , srv826881877:84 , srv85950946:84 , srv863310189:84 , srv882016226:84 , srv889543003:84 , srv899482417:84 , srv944973541:84 , srv953840699:84 , srv994656906:84 , srv998880862:84 , srv998988231:84 , srv1494045575:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-02-14 15:54:02,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-14 15:54:02,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1278446407=14, srv1561762135=25, srv1727079872=34, srv243760619=62, srv1978469510=48, srv376453246=67, srv826881877=89, srv1616566360=30, srv1684709033=33, srv2127430764=60, srv331322030=65, srv1845986435=39, srv750743354=84, srv785693732=86, srv1421930010=19, srv355229554=66, srv522457048=77, srv781580449=85, srv1628963987=32, srv472414025=75, srv1975810639=47, srv593072564=80, srv81977692=88, srv998880862=98, srv1406827548=18, srv1105557363=6, srv539318666=78, srv501357554=76, srv1589920785=26, srv1231562670=12, srv1111392893=7, srv1833824735=37, srv953840699=96, srv1471042565=22, srv1080582355=3, srv1866207076=42, srv1590029266=27, srv1925493852=45, srv2050834476=55, srv723778758=83, srv1625464073=31, srv899482417=94, srv1119858410=8, srv802500994=87, srv1011619801=0, srv2043710935=52, srv882016226=92, srv465354439=73, srv1240701023=13, srv1881157965=44, srv38115448=69, srv414213644=70, srv1595737586=29, srv1748177190=35, srv2046795662=53, srv2064454636=56, srv994656906=97, srv1228054314=11, srv377428337=68, srv592346347=79, srv863310189=91, srv673019616=82, srv1426863158=20, srv1042779744=1, srv1474695358=23, srv2099352059=59, srv120108661=9, srv1494045575=24, srv1208153922=10, srv294732698=64, srv253964902=63, srv419383482=71, srv85950946=90, srv2096314520=58, srv944973541=95, srv889543003=93, srv1838835049=38, srv1090186448=4, srv1322968277=16, srv2049784388=54, srv1065429010=2, srv1428405325=21, srv998988231=99, srv1853791311=40, srv1401764352=17, srv1871762428=43, srv2039760844=51, srv1102953625=5, srv2032489014=50, srv2002833269=49, srv1973594251=46, srv621699242=81, srv444527564=72, srv1591564338=28, srv2082442318=57, srv469145417=74, srv1810167936=36, srv1303404149=15, srv1855828918=41, srv214433328=61} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-14 15:54:02,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-14 15:54:02,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-14 15:54:02,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-14 15:54:02,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-14 15:54:02,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-14 15:54:02,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-14 15:54:02,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-14 15:54:02,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-14 15:54:02,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-14 15:54:02,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-14 15:54:02,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-14 15:54:02,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-14 15:54:02,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-14 15:54:02,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-14 15:54:02,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-14 15:54:02,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-14 15:54:02,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-14 15:54:02,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-14 15:54:02,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-14 15:54:02,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-14 15:54:02,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-14 15:54:02,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-14 15:54:02,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-14 15:54:02,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-14 15:54:02,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-14 15:54:02,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-14 15:54:02,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-14 15:54:02,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-14 15:54:02,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-14 15:54:02,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-14 15:54:02,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-14 15:54:02,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-14 15:54:02,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-14 15:54:02,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-14 15:54:02,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-14 15:54:02,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-14 15:54:02,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-14 15:54:02,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-14 15:54:02,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-14 15:54:02,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-14 15:54:02,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-14 15:54:02,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-14 15:54:02,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-14 15:54:02,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-14 15:54:02,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-14 15:54:02,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-14 15:54:02,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-14 15:54:02,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-14 15:54:02,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-14 15:54:02,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-14 15:54:02,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-14 15:54:02,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-14 15:54:02,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-14 15:54:02,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-14 15:54:02,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-14 15:54:02,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-14 15:54:02,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-14 15:54:02,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-14 15:54:02,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-14 15:54:02,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-14 15:54:02,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-14 15:54:02,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-14 15:54:02,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-14 15:54:02,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-14 15:54:02,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-14 15:54:02,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-14 15:54:02,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-14 15:54:02,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-14 15:54:02,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-14 15:54:02,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-14 15:54:02,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-14 15:54:02,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-14 15:54:02,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-14 15:54:02,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-14 15:54:02,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-14 15:54:02,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-14 15:54:02,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-14 15:54:02,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-14 15:54:02,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-14 15:54:02,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-14 15:54:02,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-14 15:54:02,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-14 15:54:02,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-14 15:54:02,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-14 15:54:02,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-14 15:54:02,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-14 15:54:02,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-14 15:54:02,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-14 15:54:02,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-14 15:54:02,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-14 15:54:02,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-14 15:54:02,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-14 15:54:02,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-14 15:54:02,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-14 15:54:02,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-14 15:54:02,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-14 15:54:02,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-14 15:54:02,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-14 15:54:02,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-14 15:54:02,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-14 15:54:02,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-14 15:54:02,618 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-14 15:54:02,619 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-14 15:54:02,619 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-14 15:54:02,958 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 491 ms to try 5000 different iterations.  Found a solution that moves 4735 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17566676768879264. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.041378539897335864, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05176784922365953, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5261111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.041378539897335864, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17066666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23941666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-14 15:54:02,994 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1119858410:82 , srv1421930010:82 , srv1833824735:82 , srv243760619:82 , srv1042779744:83 , srv1240701023:83 , srv1278446407:83 , srv1589920785:83 , srv1853791311:83 , srv38115448:83 , srv802500994:83 , srv1208153922:84 , srv1406827548:84 , srv1590029266:84 , srv1845986435:84 , srv2002833269:84 , srv2064454636:84 , srv621699242:84 , srv785693732:84 , srv998988231:84 , srv1065429010:85 , srv1080582355:85 , srv120108661:85 , srv1428405325:85 , srv1855828918:85 , srv1881157965:85 , srv1973594251:85 , srv2082442318:85 , srv214433328:85 , srv376453246:85 , srv419383482:85 , srv673019616:85 , srv81977692:85 , srv863310189:85 , srv998880862:85 , srv1011619801:86 , srv1090186448:86 , srv1102953625:86 , srv1111392893:86 , srv1228054314:86 , srv1231562670:86 , srv1303404149:86 , srv1426863158:86 , srv1471042565:86 , srv1616566360:86 , srv1727079872:86 , srv1810167936:86 , srv2043710935:86 , srv2049784388:86 , srv355229554:86 , srv377428337:86 , srv592346347:86 , srv593072564:86 , srv723778758:86 , srv826881877:86 , srv1105557363:87 , srv1322968277:87 , srv1401764352:87 , srv1595737586:87 , srv1684709033:87 , srv1748177190:87 , srv1871762428:87 , srv1925493852:87 , srv1975810639:87 , srv2032489014:87 , srv2046795662:87 , srv2050834476:87 , srv2099352059:87 , srv331322030:87 , srv444527564:87 , srv465354439:87 , srv469145417:87 , srv501357554:87 , srv522457048:87 , srv750743354:87 , srv882016226:87 , srv1561762135:88 , srv1591564338:88 , srv1625464073:88 , srv1628963987:88 , srv2039760844:88 , srv2096314520:88 , srv253964902:88 , srv294732698:88 , srv414213644:88 , srv85950946:88 , srv953840699:88 , srv1474695358:89 , srv1838835049:89 , srv1866207076:89 , srv889543003:89 , srv781580449:90 , srv1978469510:91 , srv2127430764:91 , srv899482417:91 , srv944973541:91 , srv472414025:92 , srv994656906:92 , srv539318666:93 , srv1494045575:458 }
2023-02-14 15:54:03,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-14 15:54:03,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1278446407=14, srv1561762135=25, srv1727079872=34, srv243760619=62, srv1978469510=48, srv376453246=67, srv826881877=89, srv1616566360=30, srv1684709033=33, srv2127430764=60, srv331322030=65, srv1845986435=39, srv750743354=84, srv785693732=86, srv1421930010=19, srv355229554=66, srv522457048=77, srv781580449=85, srv1628963987=32, srv472414025=75, srv1975810639=47, srv593072564=80, srv81977692=88, srv998880862=98, srv1406827548=18, srv1105557363=6, srv539318666=78, srv501357554=76, srv1589920785=26, srv1231562670=12, srv1111392893=7, srv1833824735=37, srv953840699=96, srv1471042565=22, srv1080582355=3, srv1866207076=42, srv1590029266=27, srv1925493852=45, srv2050834476=55, srv723778758=83, srv1625464073=31, srv899482417=94, srv1119858410=8, srv802500994=87, srv1011619801=0, srv2043710935=52, srv882016226=92, srv465354439=73, srv1240701023=13, srv1881157965=44, srv38115448=69, srv414213644=70, srv1595737586=29, srv1748177190=35, srv2046795662=53, srv2064454636=56, srv994656906=97, srv1228054314=11, srv377428337=68, srv592346347=79, srv863310189=91, srv673019616=82, srv1426863158=20, srv1042779744=1, srv1474695358=23, srv2099352059=59, srv120108661=9, srv1494045575=24, srv1208153922=10, srv294732698=64, srv253964902=63, srv419383482=71, srv85950946=90, srv2096314520=58, srv944973541=95, srv889543003=93, srv1838835049=38, srv1090186448=4, srv1322968277=16, srv2049784388=54, srv1065429010=2, srv1428405325=21, srv998988231=99, srv1853791311=40, srv1401764352=17, srv1871762428=43, srv2039760844=51, srv1102953625=5, srv2032489014=50, srv2002833269=49, srv1973594251=46, srv621699242=81, srv444527564=72, srv1591564338=28, srv2082442318=57, srv469145417=74, srv1810167936=36, srv1303404149=15, srv1855828918=41, srv214433328=61} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-14 15:54:03,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-14 15:54:03,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-14 15:54:03,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-14 15:54:03,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-14 15:54:03,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-14 15:54:03,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-14 15:54:03,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-14 15:54:03,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-14 15:54:03,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-14 15:54:03,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-14 15:54:03,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-14 15:54:03,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-14 15:54:03,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-14 15:54:03,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-14 15:54:03,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-14 15:54:03,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-14 15:54:03,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-14 15:54:03,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-14 15:54:03,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-14 15:54:03,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-14 15:54:03,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-14 15:54:03,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-14 15:54:03,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-14 15:54:03,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-14 15:54:03,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-14 15:54:03,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-14 15:54:03,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-14 15:54:03,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-14 15:54:03,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-14 15:54:03,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-14 15:54:03,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-14 15:54:03,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-14 15:54:03,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-14 15:54:03,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-14 15:54:03,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-14 15:54:03,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-14 15:54:03,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-14 15:54:03,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-14 15:54:03,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-14 15:54:03,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-14 15:54:03,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-14 15:54:03,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-14 15:54:03,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-14 15:54:03,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-14 15:54:03,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-14 15:54:03,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-14 15:54:03,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-14 15:54:03,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-14 15:54:03,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-14 15:54:03,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-14 15:54:03,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-14 15:54:03,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-14 15:54:03,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-14 15:54:03,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-14 15:54:03,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-14 15:54:03,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-14 15:54:03,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-14 15:54:03,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-14 15:54:03,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-14 15:54:03,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-14 15:54:03,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-14 15:54:03,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-14 15:54:03,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-14 15:54:03,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-14 15:54:03,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-14 15:54:03,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-14 15:54:03,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-14 15:54:03,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-14 15:54:03,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-14 15:54:03,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-14 15:54:03,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-14 15:54:03,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-14 15:54:03,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-14 15:54:03,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-14 15:54:03,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-14 15:54:03,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-14 15:54:03,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-14 15:54:03,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-14 15:54:03,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-14 15:54:03,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-14 15:54:03,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-14 15:54:03,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-14 15:54:03,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-14 15:54:03,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-14 15:54:03,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-14 15:54:03,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-14 15:54:03,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-14 15:54:03,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-14 15:54:03,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-14 15:54:03,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-14 15:54:03,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-14 15:54:03,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-14 15:54:03,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-14 15:54:03,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-14 15:54:03,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-14 15:54:03,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-14 15:54:03,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-14 15:54:03,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-14 15:54:03,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-14 15:54:03,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-14 15:54:03,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-14 15:54:03,205 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.041378539897335864, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05176784922365953, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.041378539897335864, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17066666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23941666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-14 15:54:03,206 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-14 15:54:03,206 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.1756336125101528, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.041378539897335864, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05176784922365953, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.041378539897335864, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17066666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23941666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-14 15:54:03,531 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 466 ms to try 5000 different iterations.  Found a solution that moves 2656 regions; Going from a computed imbalance of 0.1756336125101528 to a new imbalance of 0.01532926564243696. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021831583363869596); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02963299644171136, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2951111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021831583363869596); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012083333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04658333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-14 15:54:03,545 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv472414025:79 , srv994656906:80 , srv899482417:81 , srv944973541:81 , srv1474695358:82 , srv1684709033:82 , srv1978469510:82 , srv953840699:82 , srv2032489014:83 , srv2127430764:83 , srv539318666:83 , srv781580449:83 , srv85950946:83 , srv1102953625:84 , srv1105557363:84 , srv1231562670:84 , srv1401764352:84 , srv1561762135:84 , srv1591564338:84 , srv1595737586:84 , srv1628963987:84 , srv1727079872:84 , srv1748177190:84 , srv1810167936:84 , srv1866207076:84 , srv2046795662:84 , srv2096314520:84 , srv2099352059:84 , srv253964902:84 , srv331322030:84 , srv444527564:84 , srv469145417:84 , srv593072564:84 , srv826881877:84 , srv1011619801:85 , srv1471042565:85 , srv1590029266:85 , srv1625464073:85 , srv1838835049:85 , srv1881157965:85 , srv2039760844:85 , srv419383482:85 , srv465354439:85 , srv889543003:85 , srv1065429010:86 , srv1080582355:86 , srv1111392893:86 , srv120108661:86 , srv1228054314:86 , srv1428405325:86 , srv1616566360:86 , srv1855828918:86 , srv1925493852:86 , srv1975810639:86 , srv2049784388:86 , srv2050834476:86 , srv2082442318:86 , srv294732698:86 , srv414213644:86 , srv501357554:86 , srv522457048:86 , srv592346347:86 , srv621699242:86 , srv673019616:86 , srv750743354:86 , srv882016226:86 , srv1042779744:87 , srv1090186448:87 , srv1208153922:87 , srv1426863158:87 , srv1973594251:87 , srv2064454636:87 , srv355229554:87 , srv377428337:87 , srv723778758:87 , srv785693732:87 , srv81977692:87 , srv863310189:87 , srv1322968277:88 , srv1406827548:88 , srv1871762428:88 , srv2043710935:88 , srv214433328:88 , srv998880862:88 , srv998988231:88 , srv1278446407:89 , srv1303404149:89 , srv1833824735:89 , srv1845986435:89 , srv376453246:89 , srv802500994:89 , srv1853791311:90 , srv38115448:90 , srv1119858410:91 , srv1240701023:91 , srv1421930010:91 , srv1589920785:91 , srv2002833269:91 , srv243760619:91 , srv1494045575:510 }
2023-02-14 15:54:03,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-14 15:54:03,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1278446407=14, srv1561762135=25, srv1727079872=34, srv243760619=62, srv1978469510=48, srv376453246=67, srv826881877=89, srv1616566360=30, srv1684709033=33, srv2127430764=60, srv331322030=65, srv1845986435=39, srv750743354=84, srv785693732=86, srv1421930010=19, srv355229554=66, srv522457048=77, srv781580449=85, srv1628963987=32, srv472414025=75, srv1975810639=47, srv593072564=80, srv81977692=88, srv998880862=98, srv1406827548=18, srv1105557363=6, srv539318666=78, srv501357554=76, srv1589920785=26, srv1231562670=12, srv1111392893=7, srv1833824735=37, srv953840699=96, srv1471042565=22, srv1080582355=3, srv1866207076=42, srv1590029266=27, srv1925493852=45, srv2050834476=55, srv723778758=83, srv1625464073=31, srv899482417=94, srv1119858410=8, srv802500994=87, srv1011619801=0, srv2043710935=52, srv882016226=92, srv465354439=73, srv1240701023=13, srv1881157965=44, srv38115448=69, srv414213644=70, srv1595737586=29, srv1748177190=35, srv2046795662=53, srv2064454636=56, srv994656906=97, srv1228054314=11, srv377428337=68, srv592346347=79, srv863310189=91, srv673019616=82, srv1426863158=20, srv1042779744=1, srv1474695358=23, srv2099352059=59, srv120108661=9, srv1494045575=24, srv1208153922=10, srv294732698=64, srv253964902=63, srv419383482=71, srv85950946=90, srv2096314520=58, srv944973541=95, srv889543003=93, srv1838835049=38, srv1090186448=4, srv1322968277=16, srv2049784388=54, srv1065429010=2, srv1428405325=21, srv998988231=99, srv1853791311=40, srv1401764352=17, srv1871762428=43, srv2039760844=51, srv1102953625=5, srv2032489014=50, srv2002833269=49, srv1973594251=46, srv621699242=81, srv444527564=72, srv1591564338=28, srv2082442318=57, srv469145417=74, srv1810167936=36, srv1303404149=15, srv1855828918=41, srv214433328=61} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-14 15:54:03,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-14 15:54:03,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-14 15:54:03,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-14 15:54:03,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-14 15:54:03,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-14 15:54:03,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-14 15:54:03,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-14 15:54:03,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-14 15:54:03,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-14 15:54:03,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-14 15:54:03,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-14 15:54:03,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-14 15:54:03,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-14 15:54:03,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-14 15:54:03,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-14 15:54:03,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-14 15:54:03,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-14 15:54:03,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-14 15:54:03,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-14 15:54:03,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-14 15:54:03,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-14 15:54:03,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-14 15:54:03,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-14 15:54:03,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-14 15:54:03,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-14 15:54:03,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-14 15:54:03,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-14 15:54:03,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-14 15:54:03,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-14 15:54:03,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-14 15:54:03,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-14 15:54:03,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-14 15:54:03,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-14 15:54:03,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-14 15:54:03,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-14 15:54:03,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-14 15:54:03,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-14 15:54:03,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-14 15:54:03,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-14 15:54:03,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-14 15:54:03,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-14 15:54:03,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-14 15:54:03,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-14 15:54:03,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-14 15:54:03,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-14 15:54:03,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-14 15:54:03,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-14 15:54:03,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-14 15:54:03,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-14 15:54:03,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-14 15:54:03,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-14 15:54:03,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-14 15:54:03,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-14 15:54:03,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-14 15:54:03,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-14 15:54:03,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-14 15:54:03,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-14 15:54:03,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-14 15:54:03,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-14 15:54:03,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-14 15:54:03,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-14 15:54:03,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-14 15:54:03,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-14 15:54:03,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-14 15:54:03,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-14 15:54:03,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-14 15:54:03,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-14 15:54:03,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-14 15:54:03,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-14 15:54:03,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-14 15:54:03,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-14 15:54:03,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-14 15:54:03,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-14 15:54:03,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-14 15:54:03,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-14 15:54:03,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-14 15:54:03,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-14 15:54:03,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-14 15:54:03,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-14 15:54:03,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-14 15:54:03,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-14 15:54:03,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-14 15:54:03,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-14 15:54:03,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-14 15:54:03,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-14 15:54:03,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-14 15:54:03,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-14 15:54:03,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-14 15:54:03,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-14 15:54:03,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-14 15:54:03,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-14 15:54:03,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-14 15:54:03,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-14 15:54:03,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-14 15:54:03,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-14 15:54:03,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-14 15:54:03,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-14 15:54:03,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-14 15:54:03,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-14 15:54:03,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-14 15:54:03,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-14 15:54:03,737 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021831583363869596); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02963299644171136, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021831583363869596); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012083333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04658333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-14 15:54:03,737 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-14 15:54:03,738 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.015310667932940144, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021831583363869596); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02963299644171136, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021831583363869596); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012083333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04658333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-14 15:54:04,019 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 404 ms to try 5000 different iterations.  Found a solution that moves 989 regions; Going from a computed imbalance of 0.015310667932940144 to a new imbalance of 7.406539445703861E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008352204595864103); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009063933040070526); MoveCostFunction : (multiplier=7.0, imbalance=0.10988888888888888, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.008352204595864103); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003916666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-14 15:54:04,021 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1303404149:82 , srv1871762428:82 , srv2002833269:82 , srv539318666:82 , srv1838835049:83 , srv1090186448:84 , srv1119858410:84 , srv1228054314:84 , srv1240701023:84 , srv1421930010:84 , srv1426863158:84 , srv1589920785:84 , srv1591564338:84 , srv1628963987:84 , srv1845986435:84 , srv1853791311:84 , srv1925493852:84 , srv1978469510:84 , srv2039760844:84 , srv2043710935:84 , srv2050834476:84 , srv2127430764:84 , srv214433328:84 , srv243760619:84 , srv294732698:84 , srv376453246:84 , srv377428337:84 , srv38115448:84 , srv414213644:84 , srv465354439:84 , srv501357554:84 , srv723778758:84 , srv750743354:84 , srv781580449:84 , srv882016226:84 , srv889543003:84 , srv994656906:84 , srv1111392893:85 , srv1322968277:85 , srv1561762135:85 , srv1625464073:85 , srv1748177190:85 , srv1833824735:85 , srv1866207076:85 , srv1973594251:85 , srv1975810639:85 , srv2082442318:85 , srv355229554:85 , srv522457048:85 , srv785693732:85 , srv802500994:85 , srv81977692:85 , srv863310189:85 , srv899482417:85 , srv998880862:85 , srv998988231:85 , srv1011619801:86 , srv1065429010:86 , srv1102953625:86 , srv1105557363:86 , srv120108661:86 , srv1208153922:86 , srv1278446407:86 , srv1428405325:86 , srv1595737586:86 , srv1616566360:86 , srv1881157965:86 , srv2046795662:86 , srv2096314520:86 , srv2099352059:86 , srv253964902:86 , srv331322030:86 , srv444527564:86 , srv592346347:86 , srv673019616:86 , srv826881877:86 , srv85950946:86 , srv944973541:86 , srv953840699:86 , srv1042779744:87 , srv1080582355:87 , srv1231562670:87 , srv1401764352:87 , srv1406827548:87 , srv1471042565:87 , srv1474695358:87 , srv1590029266:87 , srv1684709033:87 , srv1727079872:87 , srv1855828918:87 , srv2032489014:87 , srv2064454636:87 , srv469145417:87 , srv472414025:87 , srv593072564:87 , srv1810167936:88 , srv2049784388:88 , srv419383482:88 , srv621699242:88 , srv1494045575:564 }
2023-02-14 15:54:04,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-14 15:54:04,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1278446407=14, srv1561762135=25, srv1727079872=34, srv243760619=62, srv1978469510=48, srv376453246=67, srv826881877=89, srv1616566360=30, srv1684709033=33, srv2127430764=60, srv331322030=65, srv1845986435=39, srv750743354=84, srv785693732=86, srv1421930010=19, srv355229554=66, srv522457048=77, srv781580449=85, srv1628963987=32, srv472414025=75, srv1975810639=47, srv593072564=80, srv81977692=88, srv998880862=98, srv1406827548=18, srv1105557363=6, srv539318666=78, srv501357554=76, srv1589920785=26, srv1231562670=12, srv1111392893=7, srv1833824735=37, srv953840699=96, srv1471042565=22, srv1080582355=3, srv1866207076=42, srv1590029266=27, srv1925493852=45, srv2050834476=55, srv723778758=83, srv1625464073=31, srv899482417=94, srv1119858410=8, srv802500994=87, srv1011619801=0, srv2043710935=52, srv882016226=92, srv465354439=73, srv1240701023=13, srv1881157965=44, srv38115448=69, srv414213644=70, srv1595737586=29, srv1748177190=35, srv2046795662=53, srv2064454636=56, srv994656906=97, srv1228054314=11, srv377428337=68, srv592346347=79, srv863310189=91, srv673019616=82, srv1426863158=20, srv1042779744=1, srv1474695358=23, srv2099352059=59, srv120108661=9, srv1494045575=24, srv1208153922=10, srv294732698=64, srv253964902=63, srv419383482=71, srv85950946=90, srv2096314520=58, srv944973541=95, srv889543003=93, srv1838835049=38, srv1090186448=4, srv1322968277=16, srv2049784388=54, srv1065429010=2, srv1428405325=21, srv998988231=99, srv1853791311=40, srv1401764352=17, srv1871762428=43, srv2039760844=51, srv1102953625=5, srv2032489014=50, srv2002833269=49, srv1973594251=46, srv621699242=81, srv444527564=72, srv1591564338=28, srv2082442318=57, srv469145417=74, srv1810167936=36, srv1303404149=15, srv1855828918=41, srv214433328=61} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-14 15:54:04,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-14 15:54:04,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-14 15:54:04,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-14 15:54:04,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-14 15:54:04,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-14 15:54:04,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-14 15:54:04,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-14 15:54:04,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-14 15:54:04,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-14 15:54:04,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-14 15:54:04,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-14 15:54:04,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-14 15:54:04,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-14 15:54:04,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-14 15:54:04,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-14 15:54:04,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-14 15:54:04,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-14 15:54:04,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-14 15:54:04,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-14 15:54:04,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-14 15:54:04,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-14 15:54:04,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-14 15:54:04,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-14 15:54:04,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-14 15:54:04,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-14 15:54:04,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-14 15:54:04,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-14 15:54:04,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-14 15:54:04,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-14 15:54:04,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-14 15:54:04,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-14 15:54:04,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-14 15:54:04,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-14 15:54:04,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-14 15:54:04,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-14 15:54:04,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-14 15:54:04,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-14 15:54:04,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-14 15:54:04,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-14 15:54:04,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-14 15:54:04,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-14 15:54:04,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-14 15:54:04,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-14 15:54:04,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-14 15:54:04,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-14 15:54:04,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-14 15:54:04,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-14 15:54:04,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-14 15:54:04,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-14 15:54:04,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-14 15:54:04,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-14 15:54:04,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-14 15:54:04,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-14 15:54:04,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-14 15:54:04,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-14 15:54:04,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-14 15:54:04,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-14 15:54:04,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-14 15:54:04,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-14 15:54:04,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-14 15:54:04,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-14 15:54:04,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-14 15:54:04,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-14 15:54:04,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-14 15:54:04,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-14 15:54:04,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-14 15:54:04,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-14 15:54:04,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-14 15:54:04,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-14 15:54:04,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-14 15:54:04,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-14 15:54:04,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-14 15:54:04,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-14 15:54:04,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-14 15:54:04,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-14 15:54:04,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-14 15:54:04,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-14 15:54:04,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-14 15:54:04,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-14 15:54:04,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-14 15:54:04,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-14 15:54:04,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-14 15:54:04,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-14 15:54:04,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-14 15:54:04,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-14 15:54:04,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-14 15:54:04,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-14 15:54:04,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-14 15:54:04,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-14 15:54:04,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-14 15:54:04,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-14 15:54:04,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-14 15:54:04,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-14 15:54:04,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-14 15:54:04,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-14 15:54:04,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-14 15:54:04,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-14 15:54:04,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-14 15:54:04,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-14 15:54:04,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-14 15:54:04,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-14 15:54:04,150 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008352204595864103); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009063933040070526); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.008352204595864103); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003916666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-14 15:54:04,150 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-14 15:54:04,150 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=7.337288185566999E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008352204595864103); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009063933040070526); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.008352204595864103); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003916666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-14 15:54:04,572 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 520 ms to try 5000 different iterations.  Found a solution that moves 267 regions; Going from a computed imbalance of 7.337288185566999E-4 to a new imbalance of 4.571129845570345E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00259499648053841); MoveCostFunction : (multiplier=7.0, imbalance=0.029666666666666668, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-14 15:54:04,573 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1322968277:82 , srv2049784388:82 , srv1406827548:83 , srv376453246:83 , srv1080582355:84 , srv1240701023:84 , srv1278446407:84 , srv1303404149:84 , srv1401764352:84 , srv1471042565:84 , srv1474695358:84 , srv1589920785:84 , srv1595737586:84 , srv1616566360:84 , srv1625464073:84 , srv1810167936:84 , srv1855828918:84 , srv1866207076:84 , srv1925493852:84 , srv1975810639:84 , srv1978469510:84 , srv2043710935:84 , srv2050834476:84 , srv2064454636:84 , srv2096314520:84 , srv2099352059:84 , srv2127430764:84 , srv253964902:84 , srv294732698:84 , srv355229554:84 , srv414213644:84 , srv419383482:84 , srv469145417:84 , srv539318666:84 , srv592346347:84 , srv621699242:84 , srv944973541:84 , srv998880862:84 , srv1011619801:85 , srv1042779744:85 , srv1065429010:85 , srv1090186448:85 , srv1105557363:85 , srv1111392893:85 , srv1119858410:85 , srv120108661:85 , srv1208153922:85 , srv1231562670:85 , srv1421930010:85 , srv1426863158:85 , srv1428405325:85 , srv1561762135:85 , srv1684709033:85 , srv1727079872:85 , srv1838835049:85 , srv1845986435:85 , srv1853791311:85 , srv1871762428:85 , srv1973594251:85 , srv2002833269:85 , srv2032489014:85 , srv2039760844:85 , srv214433328:85 , srv243760619:85 , srv331322030:85 , srv377428337:85 , srv38115448:85 , srv444527564:85 , srv472414025:85 , srv501357554:85 , srv522457048:85 , srv593072564:85 , srv723778758:85 , srv750743354:85 , srv781580449:85 , srv81977692:85 , srv863310189:85 , srv882016226:85 , srv889543003:85 , srv899482417:85 , srv998988231:85 , srv1102953625:86 , srv1228054314:86 , srv1590029266:86 , srv1591564338:86 , srv1628963987:86 , srv1748177190:86 , srv1833824735:86 , srv1881157965:86 , srv2046795662:86 , srv2082442318:86 , srv673019616:86 , srv785693732:86 , srv802500994:86 , srv826881877:86 , srv85950946:86 , srv953840699:86 , srv994656906:86 , srv465354439:87 , srv1494045575:610 }
2023-02-14 15:54:04,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-14 15:54:04,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1278446407=14, srv1561762135=25, srv1727079872=34, srv243760619=62, srv1978469510=48, srv376453246=67, srv826881877=89, srv1616566360=30, srv1684709033=33, srv2127430764=60, srv331322030=65, srv1845986435=39, srv750743354=84, srv785693732=86, srv1421930010=19, srv355229554=66, srv522457048=77, srv781580449=85, srv1628963987=32, srv472414025=75, srv1975810639=47, srv593072564=80, srv81977692=88, srv998880862=98, srv1406827548=18, srv1105557363=6, srv539318666=78, srv501357554=76, srv1589920785=26, srv1231562670=12, srv1111392893=7, srv1833824735=37, srv953840699=96, srv1471042565=22, srv1080582355=3, srv1866207076=42, srv1590029266=27, srv1925493852=45, srv2050834476=55, srv723778758=83, srv1625464073=31, srv899482417=94, srv1119858410=8, srv802500994=87, srv1011619801=0, srv2043710935=52, srv882016226=92, srv465354439=73, srv1240701023=13, srv1881157965=44, srv38115448=69, srv414213644=70, srv1595737586=29, srv1748177190=35, srv2046795662=53, srv2064454636=56, srv994656906=97, srv1228054314=11, srv377428337=68, srv592346347=79, srv863310189=91, srv673019616=82, srv1426863158=20, srv1042779744=1, srv1474695358=23, srv2099352059=59, srv120108661=9, srv1494045575=24, srv1208153922=10, srv294732698=64, srv253964902=63, srv419383482=71, srv85950946=90, srv2096314520=58, srv944973541=95, srv889543003=93, srv1838835049=38, srv1090186448=4, srv1322968277=16, srv2049784388=54, srv1065429010=2, srv1428405325=21, srv998988231=99, srv1853791311=40, srv1401764352=17, srv1871762428=43, srv2039760844=51, srv1102953625=5, srv2032489014=50, srv2002833269=49, srv1973594251=46, srv621699242=81, srv444527564=72, srv1591564338=28, srv2082442318=57, srv469145417=74, srv1810167936=36, srv1303404149=15, srv1855828918=41, srv214433328=61} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-14 15:54:04,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-14 15:54:04,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-14 15:54:04,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-14 15:54:04,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-14 15:54:04,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-14 15:54:04,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-14 15:54:04,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-14 15:54:04,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-14 15:54:04,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-14 15:54:04,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-14 15:54:04,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-14 15:54:04,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-14 15:54:04,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-14 15:54:04,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-14 15:54:04,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-14 15:54:04,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-14 15:54:04,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-14 15:54:04,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-14 15:54:04,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-14 15:54:04,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-14 15:54:04,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-14 15:54:04,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-14 15:54:04,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-14 15:54:04,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-14 15:54:04,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-14 15:54:04,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-14 15:54:04,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-14 15:54:04,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-14 15:54:04,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-14 15:54:04,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-14 15:54:04,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-14 15:54:04,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-14 15:54:04,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-14 15:54:04,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-14 15:54:04,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-14 15:54:04,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-14 15:54:04,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-14 15:54:04,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-14 15:54:04,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-14 15:54:04,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-14 15:54:04,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-14 15:54:04,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-14 15:54:04,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-14 15:54:04,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-14 15:54:04,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-14 15:54:04,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-14 15:54:04,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-14 15:54:04,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-14 15:54:04,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-14 15:54:04,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-14 15:54:04,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-14 15:54:04,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-14 15:54:04,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-14 15:54:04,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-14 15:54:04,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-14 15:54:04,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-14 15:54:04,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-14 15:54:04,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-14 15:54:04,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-14 15:54:04,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-14 15:54:04,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-14 15:54:04,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-14 15:54:04,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-14 15:54:04,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-14 15:54:04,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-14 15:54:04,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-14 15:54:04,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-14 15:54:04,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-14 15:54:04,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-14 15:54:04,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-14 15:54:04,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-14 15:54:04,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-14 15:54:04,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-14 15:54:04,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-14 15:54:04,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-14 15:54:04,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-14 15:54:04,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-14 15:54:04,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-14 15:54:04,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-14 15:54:04,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-14 15:54:04,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-14 15:54:04,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-14 15:54:04,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-14 15:54:04,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-14 15:54:04,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-14 15:54:04,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-14 15:54:04,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-14 15:54:04,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-14 15:54:04,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-14 15:54:04,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-14 15:54:04,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-14 15:54:04,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-14 15:54:04,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-14 15:54:04,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-14 15:54:04,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-14 15:54:04,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-14 15:54:04,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-14 15:54:04,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-14 15:54:04,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-14 15:54:04,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-14 15:54:04,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-14 15:54:04,736 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00259499648053841); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-14 15:54:04,736 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-14 15:54:04,736 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=4.384172449649798E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00259499648053841); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-14 15:54:04,965 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 347 ms to try 5000 different iterations.  Found a solution that moves 40 regions; Going from a computed imbalance of 4.384172449649798E-5 to a new imbalance of 9.576143648946077E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0020651563939657444); MoveCostFunction : (multiplier=7.0, imbalance=0.0044444444444444444); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-14 15:54:04,965 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv2046795662:83 , srv465354439:83 , srv472414025:83 , srv522457048:83 , srv673019616:83 , srv802500994:83 , srv85950946:83 , srv889543003:83 , srv1011619801:84 , srv1042779744:84 , srv1065429010:84 , srv1080582355:84 , srv1090186448:84 , srv1102953625:84 , srv1105557363:84 , srv1111392893:84 , srv1119858410:84 , srv120108661:84 , srv1208153922:84 , srv1228054314:84 , srv1231562670:84 , srv1240701023:84 , srv1278446407:84 , srv1322968277:84 , srv1401764352:84 , srv1406827548:84 , srv1421930010:84 , srv1426863158:84 , srv1428405325:84 , srv1471042565:84 , srv1474695358:84 , srv1561762135:84 , srv1589920785:84 , srv1590029266:84 , srv1591564338:84 , srv1616566360:84 , srv1625464073:84 , srv1628963987:84 , srv1727079872:84 , srv1748177190:84 , srv1810167936:84 , srv1833824735:84 , srv1838835049:84 , srv1845986435:84 , srv1853791311:84 , srv1855828918:84 , srv1866207076:84 , srv1871762428:84 , srv1881157965:84 , srv1973594251:84 , srv1975810639:84 , srv2002833269:84 , srv2032489014:84 , srv2039760844:84 , srv2043710935:84 , srv2049784388:84 , srv2064454636:84 , srv2082442318:84 , srv2096314520:84 , srv2127430764:84 , srv214433328:84 , srv243760619:84 , srv253964902:84 , srv294732698:84 , srv331322030:84 , srv355229554:84 , srv377428337:84 , srv38115448:84 , srv414213644:84 , srv419383482:84 , srv444527564:84 , srv469145417:84 , srv501357554:84 , srv539318666:84 , srv592346347:84 , srv593072564:84 , srv621699242:84 , srv723778758:84 , srv750743354:84 , srv781580449:84 , srv785693732:84 , srv81977692:84 , srv826881877:84 , srv863310189:84 , srv882016226:84 , srv899482417:84 , srv944973541:84 , srv953840699:84 , srv994656906:84 , srv998880862:84 , srv998988231:84 , srv1303404149:85 , srv1595737586:85 , srv1684709033:85 , srv1925493852:85 , srv1978469510:85 , srv2050834476:85 , srv2099352059:85 , srv376453246:85 , srv1494045575:684 }
2023-02-14 15:54:04,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-14 15:54:04,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1278446407=14, srv1561762135=25, srv1727079872=34, srv243760619=62, srv1978469510=48, srv376453246=67, srv826881877=89, srv1616566360=30, srv1684709033=33, srv2127430764=60, srv331322030=65, srv1845986435=39, srv750743354=84, srv785693732=86, srv1421930010=19, srv355229554=66, srv522457048=77, srv781580449=85, srv1628963987=32, srv472414025=75, srv1975810639=47, srv593072564=80, srv81977692=88, srv998880862=98, srv1406827548=18, srv1105557363=6, srv539318666=78, srv501357554=76, srv1589920785=26, srv1231562670=12, srv1111392893=7, srv1833824735=37, srv953840699=96, srv1471042565=22, srv1080582355=3, srv1866207076=42, srv1590029266=27, srv1925493852=45, srv2050834476=55, srv723778758=83, srv1625464073=31, srv899482417=94, srv1119858410=8, srv802500994=87, srv1011619801=0, srv2043710935=52, srv882016226=92, srv465354439=73, srv1240701023=13, srv1881157965=44, srv38115448=69, srv414213644=70, srv1595737586=29, srv1748177190=35, srv2046795662=53, srv2064454636=56, srv994656906=97, srv1228054314=11, srv377428337=68, srv592346347=79, srv863310189=91, srv673019616=82, srv1426863158=20, srv1042779744=1, srv1474695358=23, srv2099352059=59, srv120108661=9, srv1494045575=24, srv1208153922=10, srv294732698=64, srv253964902=63, srv419383482=71, srv85950946=90, srv2096314520=58, srv944973541=95, srv889543003=93, srv1838835049=38, srv1090186448=4, srv1322968277=16, srv2049784388=54, srv1065429010=2, srv1428405325=21, srv998988231=99, srv1853791311=40, srv1401764352=17, srv1871762428=43, srv2039760844=51, srv1102953625=5, srv2032489014=50, srv2002833269=49, srv1973594251=46, srv621699242=81, srv444527564=72, srv1591564338=28, srv2082442318=57, srv469145417=74, srv1810167936=36, srv1303404149=15, srv1855828918=41, srv214433328=61} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-14 15:54:04,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-14 15:54:04,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-14 15:54:04,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-14 15:54:04,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-14 15:54:04,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-14 15:54:04,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-14 15:54:04,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-14 15:54:04,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-14 15:54:04,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-14 15:54:04,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-14 15:54:04,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-14 15:54:04,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-14 15:54:04,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-14 15:54:04,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-14 15:54:04,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-14 15:54:04,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-14 15:54:04,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-14 15:54:04,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-14 15:54:04,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-14 15:54:04,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-14 15:54:04,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-14 15:54:04,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-14 15:54:04,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-14 15:54:04,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-14 15:54:04,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-14 15:54:04,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-14 15:54:04,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-14 15:54:04,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-14 15:54:04,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-14 15:54:04,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-14 15:54:04,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-14 15:54:04,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-14 15:54:04,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-14 15:54:04,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-14 15:54:04,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-14 15:54:04,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-14 15:54:04,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-14 15:54:04,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-14 15:54:04,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-14 15:54:04,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-14 15:54:04,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-14 15:54:04,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-14 15:54:04,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-14 15:54:04,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-14 15:54:04,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-14 15:54:04,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-14 15:54:04,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-14 15:54:04,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-14 15:54:04,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-14 15:54:04,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-14 15:54:04,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-14 15:54:04,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-14 15:54:04,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-14 15:54:04,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-14 15:54:04,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-14 15:54:04,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-14 15:54:04,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-14 15:54:04,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-14 15:54:04,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-14 15:54:04,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-14 15:54:04,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-14 15:54:04,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-14 15:54:04,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-14 15:54:04,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-14 15:54:04,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-14 15:54:04,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-14 15:54:04,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-14 15:54:04,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-14 15:54:04,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-14 15:54:04,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-14 15:54:04,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-14 15:54:04,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-14 15:54:04,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-14 15:54:04,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-14 15:54:04,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-14 15:54:04,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-14 15:54:04,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-14 15:54:04,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-14 15:54:04,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-14 15:54:04,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-14 15:54:04,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-14 15:54:04,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-14 15:54:04,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-14 15:54:04,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-14 15:54:04,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-14 15:54:04,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-14 15:54:04,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-14 15:54:04,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-14 15:54:04,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-14 15:54:04,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-14 15:54:04,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-14 15:54:04,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-14 15:54:04,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-14 15:54:04,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-14 15:54:04,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-14 15:54:04,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-14 15:54:04,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-14 15:54:04,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-14 15:54:04,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-14 15:54:04,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-14 15:54:04,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-14 15:54:05,098 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=9.296057662548253E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0020651563939657444); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-14 15:54:05,099 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv2046795662:83 , srv465354439:83 , srv472414025:83 , srv522457048:83 , srv673019616:83 , srv802500994:83 , srv85950946:83 , srv889543003:83 , srv1011619801:84 , srv1042779744:84 , srv1065429010:84 , srv1080582355:84 , srv1090186448:84 , srv1102953625:84 , srv1105557363:84 , srv1111392893:84 , srv1119858410:84 , srv120108661:84 , srv1208153922:84 , srv1228054314:84 , srv1231562670:84 , srv1240701023:84 , srv1278446407:84 , srv1322968277:84 , srv1401764352:84 , srv1406827548:84 , srv1421930010:84 , srv1426863158:84 , srv1428405325:84 , srv1471042565:84 , srv1474695358:84 , srv1561762135:84 , srv1589920785:84 , srv1590029266:84 , srv1591564338:84 , srv1616566360:84 , srv1625464073:84 , srv1628963987:84 , srv1727079872:84 , srv1748177190:84 , srv1810167936:84 , srv1833824735:84 , srv1838835049:84 , srv1845986435:84 , srv1853791311:84 , srv1855828918:84 , srv1866207076:84 , srv1871762428:84 , srv1881157965:84 , srv1973594251:84 , srv1975810639:84 , srv2002833269:84 , srv2032489014:84 , srv2039760844:84 , srv2043710935:84 , srv2049784388:84 , srv2064454636:84 , srv2082442318:84 , srv2096314520:84 , srv2127430764:84 , srv214433328:84 , srv243760619:84 , srv253964902:84 , srv294732698:84 , srv331322030:84 , srv355229554:84 , srv377428337:84 , srv38115448:84 , srv414213644:84 , srv419383482:84 , srv444527564:84 , srv469145417:84 , srv501357554:84 , srv539318666:84 , srv592346347:84 , srv593072564:84 , srv621699242:84 , srv723778758:84 , srv750743354:84 , srv781580449:84 , srv785693732:84 , srv81977692:84 , srv826881877:84 , srv863310189:84 , srv882016226:84 , srv899482417:84 , srv944973541:84 , srv953840699:84 , srv994656906:84 , srv998880862:84 , srv998988231:84 , srv1303404149:85 , srv1595737586:85 , srv1684709033:85 , srv1925493852:85 , srv1978469510:85 , srv2050834476:85 , srv2099352059:85 , srv376453246:85 , srv1494045575:684 }
2023-02-14 15:54:05,122 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=367 (was 338) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=4805 (was 5305)
2023-02-14 15:54:05,129 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=367, ProcessCount=168, AvailableMemoryMB=4805
2023-02-14 15:54:05,130 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-14 15:54:05,130 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-02-14 15:54:05,138 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1001841872:84 , srv1010336036:84 , srv1074476036:84 , srv1074806447:84 , srv1106344271:84 , srv1114408919:84 , srv1280117211:84 , srv1503896899:84 , srv1542184029:84 , srv1565562180:84 , srv1630730149:84 , srv1659974049:84 , srv170150291:84 , srv1842220346:84 , srv1875688022:84 , srv1894662680:84 , srv190940035:84 , srv2059967914:84 , srv2124794073:84 , srv2137334922:84 , srv221692825:84 , srv230045063:84 , srv303454130:84 , srv337367743:84 , srv351629150:84 , srv355043611:84 , srv420011845:84 , srv517991624:84 , srv919800454:84 , srv2029105222:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-02-14 15:54:05,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-14 15:54:05,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv355043611=26, srv1001841872=0, srv919800454=29, srv1010336036=1, srv303454130=23, srv1280117211=6, srv337367743=24, srv1074476036=2, srv230045063=22, srv1542184029=8, srv1106344271=4, srv2124794073=19, srv1074806447=3, srv2029105222=17, srv1842220346=13, srv2137334922=20, srv221692825=21, srv170150291=12, srv1894662680=15, srv190940035=16, srv517991624=28, srv1565562180=9, srv351629150=25, srv420011845=27, srv1659974049=11, srv1503896899=7, srv2059967914=18, srv1114408919=5, srv1630730149=10, srv1875688022=14} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-14 15:54:05,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-14 15:54:05,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-14 15:54:05,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-14 15:54:05,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-14 15:54:05,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-14 15:54:05,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-14 15:54:05,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-14 15:54:05,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-14 15:54:05,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-14 15:54:05,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-14 15:54:05,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-14 15:54:05,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-14 15:54:05,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-14 15:54:05,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-14 15:54:05,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-14 15:54:05,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-14 15:54:05,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-14 15:54:05,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-14 15:54:05,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-14 15:54:05,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-14 15:54:05,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-14 15:54:05,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-14 15:54:05,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-14 15:54:05,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-14 15:54:05,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-14 15:54:05,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-14 15:54:05,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-14 15:54:05,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-14 15:54:05,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-14 15:54:05,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-14 15:54:05,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-02-14 15:54:05,165 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0760141247836037, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-14 15:54:05,165 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912432561289384, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0760141247836037, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-02-14 15:56:05,151 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5812792 different iterations.  Found a solution that moves 1866 regions; Going from a computed imbalance of 0.9912432561289384 to a new imbalance of 4.3553370884861656E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6911111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-14 15:56:05,157 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1001841872:90 , srv1010336036:90 , srv1074476036:90 , srv1074806447:90 , srv1106344271:90 , srv1114408919:90 , srv1280117211:90 , srv1503896899:90 , srv1542184029:90 , srv1565562180:90 , srv1630730149:90 , srv1659974049:90 , srv170150291:90 , srv1842220346:90 , srv1875688022:90 , srv1894662680:90 , srv190940035:90 , srv2029105222:90 , srv2059967914:90 , srv2124794073:90 , srv2137334922:90 , srv221692825:90 , srv230045063:90 , srv303454130:90 , srv337367743:90 , srv351629150:90 , srv355043611:90 , srv420011845:90 , srv517991624:90 , srv919800454:90 }
2023-02-14 15:56:05,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-14 15:56:05,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv355043611=26, srv1001841872=0, srv919800454=29, srv1010336036=1, srv303454130=23, srv1280117211=6, srv337367743=24, srv1074476036=2, srv230045063=22, srv1542184029=8, srv1106344271=4, srv2124794073=19, srv1074806447=3, srv2029105222=17, srv1842220346=13, srv2137334922=20, srv221692825=21, srv170150291=12, srv1894662680=15, srv190940035=16, srv517991624=28, srv1565562180=9, srv351629150=25, srv420011845=27, srv1659974049=11, srv1503896899=7, srv2059967914=18, srv1114408919=5, srv1630730149=10, srv1875688022=14} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-14 15:56:05,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-14 15:56:05,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-14 15:56:05,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-14 15:56:05,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-14 15:56:05,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-14 15:56:05,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-14 15:56:05,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-14 15:56:05,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-14 15:56:05,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-14 15:56:05,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-14 15:56:05,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-14 15:56:05,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-14 15:56:05,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-14 15:56:05,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-14 15:56:05,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-14 15:56:05,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-14 15:56:05,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-14 15:56:05,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-14 15:56:05,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-14 15:56:05,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-14 15:56:05,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-14 15:56:05,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-14 15:56:05,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-14 15:56:05,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-14 15:56:05,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-14 15:56:05,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-14 15:56:05,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-14 15:56:05,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-14 15:56:05,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-14 15:56:05,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-14 15:56:05,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-02-14 15:56:05,175 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-14 15:56:05,175 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1001841872:90 , srv1010336036:90 , srv1074476036:90 , srv1074806447:90 , srv1106344271:90 , srv1114408919:90 , srv1280117211:90 , srv1503896899:90 , srv1542184029:90 , srv1565562180:90 , srv1630730149:90 , srv1659974049:90 , srv170150291:90 , srv1842220346:90 , srv1875688022:90 , srv1894662680:90 , srv190940035:90 , srv2029105222:90 , srv2059967914:90 , srv2124794073:90 , srv2137334922:90 , srv221692825:90 , srv230045063:90 , srv303454130:90 , srv337367743:90 , srv351629150:90 , srv355043611:90 , srv420011845:90 , srv517991624:90 , srv919800454:90 }
2023-02-14 15:56:05,187 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=239 (was 367), ProcessCount=168 (was 168), AvailableMemoryMB=4581 (was 4805)