2023-02-14 03:54:02,891 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-02-14 03:54:03,227 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-02-14 03:54:03,336 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-02-14 03:54:03,394 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-14 03:54:03,401 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-02-14 03:54:03,414 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=305, ProcessCount=171, AvailableMemoryMB=3519
2023-02-14 03:54:03,416 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-14 03:54:03,416 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-02-14 03:54:03,534 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1020945953:84 , srv1107957691:84 , srv1128910468:84 , srv1139969612:84 , srv1173730115:84 , srv1195322497:84 , srv1207913814:84 , srv1233027431:84 , srv1279304942:84 , srv1291615432:84 , srv1306900847:84 , srv1349636155:84 , srv1349731607:84 , srv1386206261:84 , srv1432808463:84 , srv1458349237:84 , srv147499249:84 , srv1526155536:84 , srv1529506007:84 , srv1530954908:84 , srv1550351013:84 , srv1556464394:84 , srv155884200:84 , srv1571911199:84 , srv1575276979:84 , srv1581898706:84 , srv1603872724:84 , srv1649930106:84 , srv1657988912:84 , srv1662487769:84 , srv1665917429:84 , srv1684390175:84 , srv1693755428:84 , srv1711321672:84 , srv1725704333:84 , srv1760773083:84 , srv1772095437:84 , srv1774599123:84 , srv1821646255:84 , srv1829013479:84 , srv1856679996:84 , srv1877264825:84 , srv1883622861:84 , srv1890761761:84 , srv1891942735:84 , srv1899855467:84 , srv191270933:84 , srv1929881319:84 , srv1936915723:84 , srv194163113:84 , srv1966700886:84 , srv1988732331:84 , srv2015110491:84 , srv2026940046:84 , srv2031597118:84 , srv2037000963:84 , srv2072769232:84 , srv2095036554:84 , srv2099139083:84 , srv2106754936:84 , srv211379109:84 , srv2114913159:84 , srv2127426464:84 , srv2136979912:84 , srv2142597021:84 , srv215829334:84 , srv234590636:84 , srv242624596:84 , srv258610745:84 , srv30215481:84 , srv306757461:84 , srv306771103:84 , srv359556672:84 , srv39799323:84 , srv414323860:84 , srv43200949:84 , srv489576152:84 , srv578143334:84 , srv597426132:84 , srv611940201:84 , srv62158349:84 , srv629803089:84 , srv638224228:84 , srv668142101:84 , srv683723546:84 , srv689738833:84 , srv73068430:84 , srv734708187:84 , srv759501052:84 , srv768135144:84 , srv793428335:84 , srv803279479:84 , srv850734492:84 , srv864096522:84 , srv895631265:84 , srv946211317:84 , srv954825688:84 , srv973676390:84 , srv985584639:84 , srv771595062:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-02-14 03:54:03,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-14 03:54:03,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1195322497=5, srv73068430=86, srv147499249=16, srv1649930106=27, srv30215481=69, srv611940201=79, srv1526155536=17, srv1556464394=21, srv306757461=70, srv1128910468=2, srv1571911199=23, srv1291615432=9, srv638224228=82, srv191270933=46, srv793428335=91, srv155884200=22, srv1966700886=50, srv1711321672=33, srv768135144=89, srv1349731607=12, srv215829334=65, srv1603872724=26, srv629803089=81, srv1693755428=32, srv1760773083=35, srv258610745=68, srv306771103=71, srv2114913159=61, srv597426132=78, srv2031597118=54, srv1432808463=14, srv1020945953=0, srv1890761761=43, srv895631265=95, srv683723546=84, srv578143334=77, srv954825688=97, srv2136979912=63, srv1856679996=40, srv1772095437=36, srv43200949=75, srv1279304942=8, srv2015110491=52, srv234590636=66, srv2072769232=56, srv2099139083=58, srv1386206261=13, srv1575276979=24, srv1684390175=31, srv1173730115=4, srv1662487769=29, srv1774599123=37, srv62158349=80, srv2026940046=53, srv1458349237=15, srv1530954908=19, srv359556672=72, srv946211317=96, srv973676390=98, srv1929881319=47, srv1988732331=51, srv1657988912=28, srv759501052=88, srv414323860=74, srv2095036554=57, srv242624596=67, srv39799323=73, srv689738833=85, srv985584639=99, srv1891942735=44, srv2127426464=62, srv1829013479=39, srv1529506007=18, srv194163113=49, srv1725704333=34, srv734708187=87, srv2142597021=64, srv864096522=94, srv1665917429=30, srv1883622861=42, srv1936915723=48, srv1349636155=11, srv1139969612=3, srv668142101=83, srv2037000963=55, srv1306900847=10, srv1581898706=25, srv489576152=76, srv771595062=90, srv1821646255=38, srv803279479=92, srv1877264825=41, srv1107957691=1, srv211379109=60, srv1207913814=6, srv850734492=93, srv1550351013=20, srv2106754936=59, srv1899855467=45, srv1233027431=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-14 03:54:03,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-14 03:54:03,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-14 03:54:03,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-14 03:54:03,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-14 03:54:03,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-14 03:54:03,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-14 03:54:03,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-14 03:54:03,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-14 03:54:03,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-14 03:54:03,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-14 03:54:03,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-14 03:54:03,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-14 03:54:03,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-14 03:54:03,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-14 03:54:03,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-14 03:54:03,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-14 03:54:03,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-14 03:54:03,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-14 03:54:03,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-14 03:54:03,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-14 03:54:03,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-14 03:54:03,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-14 03:54:03,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-14 03:54:03,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-14 03:54:03,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-14 03:54:03,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-14 03:54:03,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-14 03:54:03,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-14 03:54:03,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-14 03:54:03,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-14 03:54:03,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-14 03:54:03,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-14 03:54:03,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-14 03:54:03,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-14 03:54:03,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-14 03:54:03,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-14 03:54:03,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-14 03:54:03,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-14 03:54:03,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-14 03:54:03,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-14 03:54:03,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-14 03:54:03,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-14 03:54:03,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-14 03:54:03,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-14 03:54:03,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-14 03:54:03,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-14 03:54:03,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-14 03:54:03,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-14 03:54:03,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-14 03:54:03,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-14 03:54:03,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-14 03:54:03,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-14 03:54:03,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-14 03:54:03,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-14 03:54:03,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-14 03:54:03,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-14 03:54:03,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-14 03:54:03,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-14 03:54:03,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-14 03:54:03,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-14 03:54:03,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-14 03:54:03,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-14 03:54:03,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-14 03:54:03,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-14 03:54:03,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-14 03:54:03,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-14 03:54:03,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-14 03:54:03,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-14 03:54:03,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-14 03:54:03,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-14 03:54:03,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-14 03:54:03,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-14 03:54:03,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-14 03:54:03,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-14 03:54:03,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-14 03:54:03,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-14 03:54:03,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-14 03:54:03,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-14 03:54:03,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-14 03:54:03,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-14 03:54:03,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-14 03:54:03,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-14 03:54:03,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-14 03:54:03,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-14 03:54:03,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-14 03:54:03,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-14 03:54:03,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-14 03:54:03,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-14 03:54:03,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-14 03:54:03,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-14 03:54:03,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-14 03:54:03,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-14 03:54:03,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-14 03:54:03,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-14 03:54:03,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-14 03:54:03,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-14 03:54:03,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-14 03:54:03,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-14 03:54:03,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-14 03:54:03,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-14 03:54:03,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-14 03:54:03,796 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-14 03:54:03,796 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-14 03:54:03,797 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-14 03:54:04,097 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 433 ms to try 5000 different iterations.  Found a solution that moves 4683 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17636012455660846. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03969424230199095, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05601627148566749, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5203333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03969424230199095, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17083333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24533333333333332, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-14 03:54:04,119 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1195322497:83 , srv1306900847:83 , srv1458349237:83 , srv1556464394:83 , srv1883622861:83 , srv194163113:83 , srv2015110491:83 , srv2142597021:83 , srv306771103:83 , srv759501052:83 , srv946211317:83 , srv1291615432:84 , srv1829013479:84 , srv1856679996:84 , srv1966700886:84 , srv2114913159:84 , srv578143334:84 , srv611940201:84 , srv1020945953:85 , srv1107957691:85 , srv1279304942:85 , srv147499249:85 , srv1529506007:85 , srv1550351013:85 , srv1571911199:85 , srv1603872724:85 , srv1662487769:85 , srv1890761761:85 , srv1936915723:85 , srv2136979912:85 , srv242624596:85 , srv414323860:85 , srv62158349:85 , srv629803089:85 , srv793428335:85 , srv803279479:85 , srv850734492:85 , srv895631265:85 , srv954825688:85 , srv973676390:85 , srv1139969612:86 , srv1349636155:86 , srv1530954908:86 , srv1725704333:86 , srv1772095437:86 , srv1774599123:86 , srv1877264825:86 , srv1891942735:86 , srv1988732331:86 , srv2106754936:86 , srv211379109:86 , srv215829334:86 , srv683723546:86 , srv1128910468:87 , srv1349731607:87 , srv155884200:87 , srv1657988912:87 , srv1760773083:87 , srv1899855467:87 , srv2031597118:87 , srv2099139083:87 , srv2127426464:87 , srv30215481:87 , srv689738833:87 , srv768135144:87 , srv1207913814:88 , srv1233027431:88 , srv1526155536:88 , srv1665917429:88 , srv191270933:88 , srv1929881319:88 , srv2037000963:88 , srv2095036554:88 , srv234590636:88 , srv258610745:88 , srv306757461:88 , srv359556672:88 , srv39799323:88 , srv489576152:88 , srv597426132:88 , srv734708187:88 , srv864096522:88 , srv1432808463:89 , srv1575276979:89 , srv1581898706:89 , srv1711321672:89 , srv2026940046:89 , srv638224228:89 , srv668142101:89 , srv985584639:89 , srv1173730115:90 , srv1386206261:90 , srv2072769232:90 , srv43200949:90 , srv1649930106:91 , srv1693755428:91 , srv1821646255:91 , srv73068430:91 , srv1684390175:92 , srv771595062:443 }
2023-02-14 03:54:04,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-14 03:54:04,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1195322497=5, srv73068430=86, srv147499249=16, srv1649930106=27, srv30215481=69, srv611940201=79, srv1526155536=17, srv1556464394=21, srv306757461=70, srv1128910468=2, srv1571911199=23, srv1291615432=9, srv638224228=82, srv191270933=46, srv793428335=91, srv155884200=22, srv1966700886=50, srv1711321672=33, srv768135144=89, srv1349731607=12, srv215829334=65, srv1603872724=26, srv629803089=81, srv1693755428=32, srv1760773083=35, srv258610745=68, srv306771103=71, srv2114913159=61, srv597426132=78, srv2031597118=54, srv1432808463=14, srv1020945953=0, srv1890761761=43, srv895631265=95, srv683723546=84, srv578143334=77, srv954825688=97, srv2136979912=63, srv1856679996=40, srv1772095437=36, srv43200949=75, srv1279304942=8, srv2015110491=52, srv234590636=66, srv2072769232=56, srv2099139083=58, srv1386206261=13, srv1575276979=24, srv1684390175=31, srv1173730115=4, srv1662487769=29, srv1774599123=37, srv62158349=80, srv2026940046=53, srv1458349237=15, srv1530954908=19, srv359556672=72, srv946211317=96, srv973676390=98, srv1929881319=47, srv1988732331=51, srv1657988912=28, srv759501052=88, srv414323860=74, srv2095036554=57, srv242624596=67, srv39799323=73, srv689738833=85, srv985584639=99, srv1891942735=44, srv2127426464=62, srv1829013479=39, srv1529506007=18, srv194163113=49, srv1725704333=34, srv734708187=87, srv2142597021=64, srv864096522=94, srv1665917429=30, srv1883622861=42, srv1936915723=48, srv1349636155=11, srv1139969612=3, srv668142101=83, srv2037000963=55, srv1306900847=10, srv1581898706=25, srv489576152=76, srv771595062=90, srv1821646255=38, srv803279479=92, srv1877264825=41, srv1107957691=1, srv211379109=60, srv1207913814=6, srv850734492=93, srv1550351013=20, srv2106754936=59, srv1899855467=45, srv1233027431=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-14 03:54:04,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-14 03:54:04,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-14 03:54:04,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-14 03:54:04,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-14 03:54:04,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-14 03:54:04,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-14 03:54:04,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-14 03:54:04,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-14 03:54:04,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-14 03:54:04,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-14 03:54:04,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-14 03:54:04,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-14 03:54:04,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-14 03:54:04,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-14 03:54:04,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-14 03:54:04,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-14 03:54:04,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-14 03:54:04,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-14 03:54:04,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-14 03:54:04,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-14 03:54:04,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-14 03:54:04,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-14 03:54:04,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-14 03:54:04,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-14 03:54:04,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-14 03:54:04,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-14 03:54:04,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-14 03:54:04,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-14 03:54:04,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-14 03:54:04,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-14 03:54:04,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-14 03:54:04,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-14 03:54:04,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-14 03:54:04,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-14 03:54:04,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-14 03:54:04,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-14 03:54:04,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-14 03:54:04,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-14 03:54:04,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-14 03:54:04,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-14 03:54:04,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-14 03:54:04,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-14 03:54:04,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-14 03:54:04,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-14 03:54:04,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-14 03:54:04,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-14 03:54:04,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-14 03:54:04,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-14 03:54:04,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-14 03:54:04,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-14 03:54:04,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-14 03:54:04,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-14 03:54:04,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-14 03:54:04,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-14 03:54:04,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-14 03:54:04,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-14 03:54:04,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-14 03:54:04,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-14 03:54:04,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-14 03:54:04,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-14 03:54:04,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-14 03:54:04,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-14 03:54:04,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-14 03:54:04,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-14 03:54:04,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-14 03:54:04,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-14 03:54:04,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-14 03:54:04,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-14 03:54:04,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-14 03:54:04,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-14 03:54:04,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-14 03:54:04,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-14 03:54:04,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-14 03:54:04,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-14 03:54:04,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-14 03:54:04,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-14 03:54:04,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-14 03:54:04,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-14 03:54:04,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-14 03:54:04,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-14 03:54:04,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-14 03:54:04,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-14 03:54:04,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-14 03:54:04,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-14 03:54:04,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-14 03:54:04,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-14 03:54:04,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-14 03:54:04,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-14 03:54:04,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-14 03:54:04,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-14 03:54:04,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-14 03:54:04,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-14 03:54:04,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-14 03:54:04,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-14 03:54:04,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-14 03:54:04,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-14 03:54:04,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-14 03:54:04,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-14 03:54:04,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-14 03:54:04,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-14 03:54:04,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-14 03:54:04,283 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03969424230199095, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05601627148566749, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03969424230199095, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17083333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24533333333333332, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-14 03:54:04,283 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-14 03:54:04,284 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17632733348975094, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03969424230199095, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05601627148566749, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03969424230199095, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17083333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24533333333333332, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-14 03:54:04,567 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 408 ms to try 5000 different iterations.  Found a solution that moves 2631 regions; Going from a computed imbalance of 0.17632733348975094 to a new imbalance of 0.015849661980339883. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.019376102440989418); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.031240711301900752, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29233333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.019376102440989418); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01275); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04575, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-14 03:54:04,576 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1821646255:79 , srv1173730115:80 , srv1386206261:80 , srv1684390175:81 , srv1693755428:81 , srv1432808463:82 , srv73068430:82 , srv1207913814:83 , srv155884200:83 , srv1575276979:83 , srv1649930106:83 , srv1665917429:83 , srv2072769232:83 , srv39799323:83 , srv43200949:83 , srv1349731607:84 , srv1581898706:84 , srv1711321672:84 , srv1760773083:84 , srv1891942735:84 , srv191270933:84 , srv1929881319:84 , srv2026940046:84 , srv2127426464:84 , srv234590636:84 , srv359556672:84 , srv638224228:84 , srv668142101:84 , srv864096522:84 , srv973676390:84 , srv1128910468:85 , srv1233027431:85 , srv1349636155:85 , srv1526155536:85 , srv1530954908:85 , srv1603872724:85 , srv1774599123:85 , srv1890761761:85 , srv1899855467:85 , srv2037000963:85 , srv2099139083:85 , srv211379109:85 , srv258610745:85 , srv306757461:85 , srv734708187:85 , srv768135144:85 , srv793428335:85 , srv985584639:85 , srv1020945953:86 , srv1139969612:86 , srv1529506007:86 , srv1662487769:86 , srv1725704333:86 , srv1772095437:86 , srv1856679996:86 , srv2095036554:86 , srv30215481:86 , srv489576152:86 , srv597426132:86 , srv62158349:86 , srv683723546:86 , srv895631265:86 , srv954825688:86 , srv1279304942:87 , srv1306900847:87 , srv1571911199:87 , srv1829013479:87 , srv1877264825:87 , srv2015110491:87 , srv2031597118:87 , srv2106754936:87 , srv2142597021:87 , srv242624596:87 , srv689738833:87 , srv1107957691:88 , srv1291615432:88 , srv1458349237:88 , srv1550351013:88 , srv1657988912:88 , srv1883622861:88 , srv1988732331:88 , srv215829334:88 , srv306771103:88 , srv414323860:88 , srv611940201:88 , srv803279479:88 , srv850734492:88 , srv2136979912:89 , srv578143334:89 , srv1966700886:90 , srv2114913159:90 , srv629803089:90 , srv147499249:91 , srv1556464394:91 , srv1936915723:91 , srv759501052:91 , srv946211317:91 , srv1195322497:92 , srv194163113:92 , srv771595062:503 }
2023-02-14 03:54:04,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-14 03:54:04,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1195322497=5, srv73068430=86, srv147499249=16, srv1649930106=27, srv30215481=69, srv611940201=79, srv1526155536=17, srv1556464394=21, srv306757461=70, srv1128910468=2, srv1571911199=23, srv1291615432=9, srv638224228=82, srv191270933=46, srv793428335=91, srv155884200=22, srv1966700886=50, srv1711321672=33, srv768135144=89, srv1349731607=12, srv215829334=65, srv1603872724=26, srv629803089=81, srv1693755428=32, srv1760773083=35, srv258610745=68, srv306771103=71, srv2114913159=61, srv597426132=78, srv2031597118=54, srv1432808463=14, srv1020945953=0, srv1890761761=43, srv895631265=95, srv683723546=84, srv578143334=77, srv954825688=97, srv2136979912=63, srv1856679996=40, srv1772095437=36, srv43200949=75, srv1279304942=8, srv2015110491=52, srv234590636=66, srv2072769232=56, srv2099139083=58, srv1386206261=13, srv1575276979=24, srv1684390175=31, srv1173730115=4, srv1662487769=29, srv1774599123=37, srv62158349=80, srv2026940046=53, srv1458349237=15, srv1530954908=19, srv359556672=72, srv946211317=96, srv973676390=98, srv1929881319=47, srv1988732331=51, srv1657988912=28, srv759501052=88, srv414323860=74, srv2095036554=57, srv242624596=67, srv39799323=73, srv689738833=85, srv985584639=99, srv1891942735=44, srv2127426464=62, srv1829013479=39, srv1529506007=18, srv194163113=49, srv1725704333=34, srv734708187=87, srv2142597021=64, srv864096522=94, srv1665917429=30, srv1883622861=42, srv1936915723=48, srv1349636155=11, srv1139969612=3, srv668142101=83, srv2037000963=55, srv1306900847=10, srv1581898706=25, srv489576152=76, srv771595062=90, srv1821646255=38, srv803279479=92, srv1877264825=41, srv1107957691=1, srv211379109=60, srv1207913814=6, srv850734492=93, srv1550351013=20, srv2106754936=59, srv1899855467=45, srv1233027431=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-14 03:54:04,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-14 03:54:04,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-14 03:54:04,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-14 03:54:04,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-14 03:54:04,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-14 03:54:04,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-14 03:54:04,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-14 03:54:04,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-14 03:54:04,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-14 03:54:04,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-14 03:54:04,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-14 03:54:04,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-14 03:54:04,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-14 03:54:04,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-14 03:54:04,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-14 03:54:04,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-14 03:54:04,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-14 03:54:04,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-14 03:54:04,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-14 03:54:04,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-14 03:54:04,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-14 03:54:04,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-14 03:54:04,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-14 03:54:04,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-14 03:54:04,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-14 03:54:04,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-14 03:54:04,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-14 03:54:04,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-14 03:54:04,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-14 03:54:04,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-14 03:54:04,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-14 03:54:04,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-14 03:54:04,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-14 03:54:04,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-14 03:54:04,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-14 03:54:04,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-14 03:54:04,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-14 03:54:04,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-14 03:54:04,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-14 03:54:04,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-14 03:54:04,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-14 03:54:04,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-14 03:54:04,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-14 03:54:04,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-14 03:54:04,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-14 03:54:04,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-14 03:54:04,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-14 03:54:04,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-14 03:54:04,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-14 03:54:04,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-14 03:54:04,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-14 03:54:04,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-14 03:54:04,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-14 03:54:04,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-14 03:54:04,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-14 03:54:04,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-14 03:54:04,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-14 03:54:04,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-14 03:54:04,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-14 03:54:04,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-14 03:54:04,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-14 03:54:04,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-14 03:54:04,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-14 03:54:04,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-14 03:54:04,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-14 03:54:04,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-14 03:54:04,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-14 03:54:04,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-14 03:54:04,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-14 03:54:04,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-14 03:54:04,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-14 03:54:04,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-14 03:54:04,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-14 03:54:04,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-14 03:54:04,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-14 03:54:04,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-14 03:54:04,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-14 03:54:04,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-14 03:54:04,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-14 03:54:04,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-14 03:54:04,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-14 03:54:04,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-14 03:54:04,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-14 03:54:04,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-14 03:54:04,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-14 03:54:04,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-14 03:54:04,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-14 03:54:04,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-14 03:54:04,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-14 03:54:04,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-14 03:54:04,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-14 03:54:04,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-14 03:54:04,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-14 03:54:04,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-14 03:54:04,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-14 03:54:04,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-14 03:54:04,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-14 03:54:04,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-14 03:54:04,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-14 03:54:04,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-14 03:54:04,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-14 03:54:04,752 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.019376102440989418); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.031240711301900752, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.019376102440989418); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01275); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04575, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-14 03:54:04,752 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-14 03:54:04,753 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.015831239324584565, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.019376102440989418); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.031240711301900752, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.019376102440989418); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01275); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04575, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-14 03:54:05,069 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 446 ms to try 5000 different iterations.  Found a solution that moves 1008 regions; Going from a computed imbalance of 0.015831239324584565 to a new imbalance of 7.073481092982218E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007907369952363212); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.01044465935734187); MoveCostFunction : (multiplier=7.0, imbalance=0.112, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007907369952363212); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004333333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-14 03:54:05,071 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv147499249:81 , srv1657988912:81 , srv194163113:81 , srv1195322497:82 , srv629803089:82 , srv1107957691:83 , srv1550351013:83 , srv1556464394:83 , srv1649930106:83 , srv1877264825:83 , srv1936915723:83 , srv2026940046:83 , srv2031597118:83 , srv2114913159:83 , srv2136979912:83 , srv215829334:83 , srv43200949:83 , srv597426132:83 , srv638224228:83 , srv1575276979:84 , srv1581898706:84 , srv1711321672:84 , srv1966700886:84 , srv2095036554:84 , srv234590636:84 , srv258610745:84 , srv30215481:84 , srv414323860:84 , srv489576152:84 , srv578143334:84 , srv689738833:84 , srv759501052:84 , srv985584639:84 , srv1020945953:85 , srv1128910468:85 , srv1291615432:85 , srv1526155536:85 , srv1725704333:85 , srv1772095437:85 , srv1899855467:85 , srv191270933:85 , srv1988732331:85 , srv2037000963:85 , srv2072769232:85 , srv2106754936:85 , srv306757461:85 , srv359556672:85 , srv668142101:85 , srv683723546:85 , srv73068430:85 , srv734708187:85 , srv768135144:85 , srv850734492:85 , srv864096522:85 , srv946211317:85 , srv1233027431:86 , srv1279304942:86 , srv1349636155:86 , srv1349731607:86 , srv1386206261:86 , srv1432808463:86 , srv1458349237:86 , srv1529506007:86 , srv1530954908:86 , srv155884200:86 , srv1571911199:86 , srv1662487769:86 , srv1684390175:86 , srv1693755428:86 , srv1760773083:86 , srv1774599123:86 , srv1856679996:86 , srv1883622861:86 , srv1929881319:86 , srv2015110491:86 , srv2099139083:86 , srv211379109:86 , srv2127426464:86 , srv242624596:86 , srv306771103:86 , srv611940201:86 , srv62158349:86 , srv803279479:86 , srv954825688:86 , srv1139969612:87 , srv1173730115:87 , srv1207913814:87 , srv1306900847:87 , srv1603872724:87 , srv1665917429:87 , srv1821646255:87 , srv1829013479:87 , srv1890761761:87 , srv1891942735:87 , srv39799323:87 , srv895631265:87 , srv2142597021:88 , srv793428335:88 , srv973676390:89 , srv771595062:582 }
2023-02-14 03:54:05,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-14 03:54:05,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1195322497=5, srv73068430=86, srv147499249=16, srv1649930106=27, srv30215481=69, srv611940201=79, srv1526155536=17, srv1556464394=21, srv306757461=70, srv1128910468=2, srv1571911199=23, srv1291615432=9, srv638224228=82, srv191270933=46, srv793428335=91, srv155884200=22, srv1966700886=50, srv1711321672=33, srv768135144=89, srv1349731607=12, srv215829334=65, srv1603872724=26, srv629803089=81, srv1693755428=32, srv1760773083=35, srv258610745=68, srv306771103=71, srv2114913159=61, srv597426132=78, srv2031597118=54, srv1432808463=14, srv1020945953=0, srv1890761761=43, srv895631265=95, srv683723546=84, srv578143334=77, srv954825688=97, srv2136979912=63, srv1856679996=40, srv1772095437=36, srv43200949=75, srv1279304942=8, srv2015110491=52, srv234590636=66, srv2072769232=56, srv2099139083=58, srv1386206261=13, srv1575276979=24, srv1684390175=31, srv1173730115=4, srv1662487769=29, srv1774599123=37, srv62158349=80, srv2026940046=53, srv1458349237=15, srv1530954908=19, srv359556672=72, srv946211317=96, srv973676390=98, srv1929881319=47, srv1988732331=51, srv1657988912=28, srv759501052=88, srv414323860=74, srv2095036554=57, srv242624596=67, srv39799323=73, srv689738833=85, srv985584639=99, srv1891942735=44, srv2127426464=62, srv1829013479=39, srv1529506007=18, srv194163113=49, srv1725704333=34, srv734708187=87, srv2142597021=64, srv864096522=94, srv1665917429=30, srv1883622861=42, srv1936915723=48, srv1349636155=11, srv1139969612=3, srv668142101=83, srv2037000963=55, srv1306900847=10, srv1581898706=25, srv489576152=76, srv771595062=90, srv1821646255=38, srv803279479=92, srv1877264825=41, srv1107957691=1, srv211379109=60, srv1207913814=6, srv850734492=93, srv1550351013=20, srv2106754936=59, srv1899855467=45, srv1233027431=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-14 03:54:05,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-14 03:54:05,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-14 03:54:05,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-14 03:54:05,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-14 03:54:05,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-14 03:54:05,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-14 03:54:05,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-14 03:54:05,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-14 03:54:05,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-14 03:54:05,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-14 03:54:05,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-14 03:54:05,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-14 03:54:05,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-14 03:54:05,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-14 03:54:05,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-14 03:54:05,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-14 03:54:05,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-14 03:54:05,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-14 03:54:05,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-14 03:54:05,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-14 03:54:05,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-14 03:54:05,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-14 03:54:05,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-14 03:54:05,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-14 03:54:05,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-14 03:54:05,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-14 03:54:05,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-14 03:54:05,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-14 03:54:05,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-14 03:54:05,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-14 03:54:05,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-14 03:54:05,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-14 03:54:05,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-14 03:54:05,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-14 03:54:05,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-14 03:54:05,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-14 03:54:05,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-14 03:54:05,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-14 03:54:05,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-14 03:54:05,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-14 03:54:05,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-14 03:54:05,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-14 03:54:05,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-14 03:54:05,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-14 03:54:05,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-14 03:54:05,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-14 03:54:05,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-14 03:54:05,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-14 03:54:05,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-14 03:54:05,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-14 03:54:05,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-14 03:54:05,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-14 03:54:05,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-14 03:54:05,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-14 03:54:05,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-14 03:54:05,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-14 03:54:05,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-14 03:54:05,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-14 03:54:05,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-14 03:54:05,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-14 03:54:05,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-14 03:54:05,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-14 03:54:05,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-14 03:54:05,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-14 03:54:05,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-14 03:54:05,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-14 03:54:05,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-14 03:54:05,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-14 03:54:05,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-14 03:54:05,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-14 03:54:05,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-14 03:54:05,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-14 03:54:05,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-14 03:54:05,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-14 03:54:05,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-14 03:54:05,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-14 03:54:05,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-14 03:54:05,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-14 03:54:05,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-14 03:54:05,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-14 03:54:05,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-14 03:54:05,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-14 03:54:05,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-14 03:54:05,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-14 03:54:05,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-14 03:54:05,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-14 03:54:05,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-14 03:54:05,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-14 03:54:05,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-14 03:54:05,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-14 03:54:05,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-14 03:54:05,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-14 03:54:05,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-14 03:54:05,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-14 03:54:05,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-14 03:54:05,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-14 03:54:05,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-14 03:54:05,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-14 03:54:05,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-14 03:54:05,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-14 03:54:05,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-14 03:54:05,232 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007907369952363212); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.01044465935734187); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007907369952363212); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004333333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-14 03:54:05,232 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-14 03:54:05,233 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=7.002899424409967E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007907369952363212); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.01044465935734187); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007907369952363212); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004333333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-14 03:54:05,616 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 493 ms to try 5000 different iterations.  Found a solution that moves 265 regions; Going from a computed imbalance of 7.002899424409967E-4 to a new imbalance of 3.358715999463854E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0015713484026367722); MoveCostFunction : (multiplier=7.0, imbalance=0.029444444444444443, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-14 03:54:05,617 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1233027431:83 , srv1684390175:83 , srv1936915723:83 , srv803279479:83 , srv946211317:83 , srv973676390:83 , srv1139969612:84 , srv1207913814:84 , srv1279304942:84 , srv1526155536:84 , srv1529506007:84 , srv1571911199:84 , srv1665917429:84 , srv1883622861:84 , srv1891942735:84 , srv1929881319:84 , srv1966700886:84 , srv1988732331:84 , srv2037000963:84 , srv2072769232:84 , srv2095036554:84 , srv2099139083:84 , srv2106754936:84 , srv2142597021:84 , srv215829334:84 , srv242624596:84 , srv306757461:84 , srv39799323:84 , srv489576152:84 , srv611940201:84 , srv668142101:84 , srv689738833:84 , srv73068430:84 , srv734708187:84 , srv793428335:84 , srv864096522:84 , srv895631265:84 , srv985584639:84 , srv1128910468:85 , srv1173730115:85 , srv1195322497:85 , srv1291615432:85 , srv1306900847:85 , srv1349636155:85 , srv1349731607:85 , srv1432808463:85 , srv1458349237:85 , srv147499249:85 , srv1530954908:85 , srv1556464394:85 , srv155884200:85 , srv1581898706:85 , srv1603872724:85 , srv1649930106:85 , srv1662487769:85 , srv1693755428:85 , srv1725704333:85 , srv1760773083:85 , srv1772095437:85 , srv1774599123:85 , srv1821646255:85 , srv1829013479:85 , srv1890761761:85 , srv1899855467:85 , srv191270933:85 , srv2031597118:85 , srv211379109:85 , srv2114913159:85 , srv2136979912:85 , srv258610745:85 , srv30215481:85 , srv306771103:85 , srv359556672:85 , srv414323860:85 , srv578143334:85 , srv597426132:85 , srv62158349:85 , srv629803089:85 , srv638224228:85 , srv683723546:85 , srv759501052:85 , srv768135144:85 , srv850734492:85 , srv954825688:85 , srv1020945953:86 , srv1107957691:86 , srv1386206261:86 , srv1550351013:86 , srv1575276979:86 , srv1657988912:86 , srv1711321672:86 , srv1856679996:86 , srv1877264825:86 , srv194163113:86 , srv2015110491:86 , srv2026940046:86 , srv2127426464:86 , srv234590636:86 , srv43200949:86 , srv771595062:614 }
2023-02-14 03:54:05,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-14 03:54:05,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1195322497=5, srv73068430=86, srv147499249=16, srv1649930106=27, srv30215481=69, srv611940201=79, srv1526155536=17, srv1556464394=21, srv306757461=70, srv1128910468=2, srv1571911199=23, srv1291615432=9, srv638224228=82, srv191270933=46, srv793428335=91, srv155884200=22, srv1966700886=50, srv1711321672=33, srv768135144=89, srv1349731607=12, srv215829334=65, srv1603872724=26, srv629803089=81, srv1693755428=32, srv1760773083=35, srv258610745=68, srv306771103=71, srv2114913159=61, srv597426132=78, srv2031597118=54, srv1432808463=14, srv1020945953=0, srv1890761761=43, srv895631265=95, srv683723546=84, srv578143334=77, srv954825688=97, srv2136979912=63, srv1856679996=40, srv1772095437=36, srv43200949=75, srv1279304942=8, srv2015110491=52, srv234590636=66, srv2072769232=56, srv2099139083=58, srv1386206261=13, srv1575276979=24, srv1684390175=31, srv1173730115=4, srv1662487769=29, srv1774599123=37, srv62158349=80, srv2026940046=53, srv1458349237=15, srv1530954908=19, srv359556672=72, srv946211317=96, srv973676390=98, srv1929881319=47, srv1988732331=51, srv1657988912=28, srv759501052=88, srv414323860=74, srv2095036554=57, srv242624596=67, srv39799323=73, srv689738833=85, srv985584639=99, srv1891942735=44, srv2127426464=62, srv1829013479=39, srv1529506007=18, srv194163113=49, srv1725704333=34, srv734708187=87, srv2142597021=64, srv864096522=94, srv1665917429=30, srv1883622861=42, srv1936915723=48, srv1349636155=11, srv1139969612=3, srv668142101=83, srv2037000963=55, srv1306900847=10, srv1581898706=25, srv489576152=76, srv771595062=90, srv1821646255=38, srv803279479=92, srv1877264825=41, srv1107957691=1, srv211379109=60, srv1207913814=6, srv850734492=93, srv1550351013=20, srv2106754936=59, srv1899855467=45, srv1233027431=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-14 03:54:05,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-14 03:54:05,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-14 03:54:05,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-14 03:54:05,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-14 03:54:05,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-14 03:54:05,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-14 03:54:05,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-14 03:54:05,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-14 03:54:05,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-14 03:54:05,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-14 03:54:05,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-14 03:54:05,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-14 03:54:05,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-14 03:54:05,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-14 03:54:05,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-14 03:54:05,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-14 03:54:05,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-14 03:54:05,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-14 03:54:05,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-14 03:54:05,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-14 03:54:05,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-14 03:54:05,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-14 03:54:05,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-14 03:54:05,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-14 03:54:05,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-14 03:54:05,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-14 03:54:05,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-14 03:54:05,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-14 03:54:05,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-14 03:54:05,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-14 03:54:05,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-14 03:54:05,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-14 03:54:05,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-14 03:54:05,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-14 03:54:05,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-14 03:54:05,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-14 03:54:05,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-14 03:54:05,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-14 03:54:05,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-14 03:54:05,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-14 03:54:05,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-14 03:54:05,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-14 03:54:05,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-14 03:54:05,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-14 03:54:05,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-14 03:54:05,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-14 03:54:05,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-14 03:54:05,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-14 03:54:05,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-14 03:54:05,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-14 03:54:05,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-14 03:54:05,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-14 03:54:05,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-14 03:54:05,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-14 03:54:05,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-14 03:54:05,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-14 03:54:05,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-14 03:54:05,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-14 03:54:05,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-14 03:54:05,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-14 03:54:05,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-14 03:54:05,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-14 03:54:05,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-14 03:54:05,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-14 03:54:05,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-14 03:54:05,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-14 03:54:05,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-14 03:54:05,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-14 03:54:05,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-14 03:54:05,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-14 03:54:05,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-14 03:54:05,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-14 03:54:05,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-14 03:54:05,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-14 03:54:05,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-14 03:54:05,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-14 03:54:05,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-14 03:54:05,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-14 03:54:05,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-14 03:54:05,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-14 03:54:05,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-14 03:54:05,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-14 03:54:05,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-14 03:54:05,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-14 03:54:05,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-14 03:54:05,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-14 03:54:05,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-14 03:54:05,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-14 03:54:05,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-14 03:54:05,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-14 03:54:05,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-14 03:54:05,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-14 03:54:05,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-14 03:54:05,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-14 03:54:05,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-14 03:54:05,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-14 03:54:05,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-14 03:54:05,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-14 03:54:05,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-14 03:54:05,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-14 03:54:05,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-14 03:54:05,795 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0015713484026367722); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-14 03:54:05,795 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-14 03:54:05,796 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=3.1731590334752956E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0015713484026367722); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-14 03:54:06,004 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 339 ms to try 5000 different iterations.  Found a solution that moves 20 regions; Going from a computed imbalance of 3.1731590334752956E-5 to a new imbalance of 6.88411204703086E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014982219165849825); MoveCostFunction : (multiplier=7.0, imbalance=0.0022222222222222222); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-14 03:54:06,004 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1139969612:83 , srv1693755428:83 , srv1711321672:83 , srv1829013479:83 , srv1988732331:83 , srv2127426464:83 , srv759501052:83 , srv850734492:83 , srv1020945953:84 , srv1107957691:84 , srv1128910468:84 , srv1173730115:84 , srv1195322497:84 , srv1207913814:84 , srv1233027431:84 , srv1279304942:84 , srv1291615432:84 , srv1306900847:84 , srv1349636155:84 , srv1349731607:84 , srv1386206261:84 , srv1432808463:84 , srv1458349237:84 , srv147499249:84 , srv1526155536:84 , srv1530954908:84 , srv1550351013:84 , srv1556464394:84 , srv1571911199:84 , srv1575276979:84 , srv1581898706:84 , srv1603872724:84 , srv1649930106:84 , srv1657988912:84 , srv1662487769:84 , srv1665917429:84 , srv1684390175:84 , srv1725704333:84 , srv1760773083:84 , srv1772095437:84 , srv1774599123:84 , srv1821646255:84 , srv1856679996:84 , srv1877264825:84 , srv1890761761:84 , srv1899855467:84 , srv191270933:84 , srv1929881319:84 , srv1936915723:84 , srv194163113:84 , srv1966700886:84 , srv2015110491:84 , srv2026940046:84 , srv2031597118:84 , srv2037000963:84 , srv2072769232:84 , srv2095036554:84 , srv2099139083:84 , srv2106754936:84 , srv211379109:84 , srv2114913159:84 , srv2136979912:84 , srv2142597021:84 , srv234590636:84 , srv242624596:84 , srv258610745:84 , srv30215481:84 , srv306757461:84 , srv306771103:84 , srv359556672:84 , srv39799323:84 , srv414323860:84 , srv43200949:84 , srv489576152:84 , srv578143334:84 , srv597426132:84 , srv611940201:84 , srv62158349:84 , srv629803089:84 , srv668142101:84 , srv683723546:84 , srv689738833:84 , srv73068430:84 , srv734708187:84 , srv768135144:84 , srv793428335:84 , srv803279479:84 , srv895631265:84 , srv946211317:84 , srv954825688:84 , srv985584639:84 , srv1529506007:85 , srv155884200:85 , srv1883622861:85 , srv1891942735:85 , srv215829334:85 , srv638224228:85 , srv864096522:85 , srv973676390:85 , srv771595062:684 }
2023-02-14 03:54:06,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-14 03:54:06,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1195322497=5, srv73068430=86, srv147499249=16, srv1649930106=27, srv30215481=69, srv611940201=79, srv1526155536=17, srv1556464394=21, srv306757461=70, srv1128910468=2, srv1571911199=23, srv1291615432=9, srv638224228=82, srv191270933=46, srv793428335=91, srv155884200=22, srv1966700886=50, srv1711321672=33, srv768135144=89, srv1349731607=12, srv215829334=65, srv1603872724=26, srv629803089=81, srv1693755428=32, srv1760773083=35, srv258610745=68, srv306771103=71, srv2114913159=61, srv597426132=78, srv2031597118=54, srv1432808463=14, srv1020945953=0, srv1890761761=43, srv895631265=95, srv683723546=84, srv578143334=77, srv954825688=97, srv2136979912=63, srv1856679996=40, srv1772095437=36, srv43200949=75, srv1279304942=8, srv2015110491=52, srv234590636=66, srv2072769232=56, srv2099139083=58, srv1386206261=13, srv1575276979=24, srv1684390175=31, srv1173730115=4, srv1662487769=29, srv1774599123=37, srv62158349=80, srv2026940046=53, srv1458349237=15, srv1530954908=19, srv359556672=72, srv946211317=96, srv973676390=98, srv1929881319=47, srv1988732331=51, srv1657988912=28, srv759501052=88, srv414323860=74, srv2095036554=57, srv242624596=67, srv39799323=73, srv689738833=85, srv985584639=99, srv1891942735=44, srv2127426464=62, srv1829013479=39, srv1529506007=18, srv194163113=49, srv1725704333=34, srv734708187=87, srv2142597021=64, srv864096522=94, srv1665917429=30, srv1883622861=42, srv1936915723=48, srv1349636155=11, srv1139969612=3, srv668142101=83, srv2037000963=55, srv1306900847=10, srv1581898706=25, srv489576152=76, srv771595062=90, srv1821646255=38, srv803279479=92, srv1877264825=41, srv1107957691=1, srv211379109=60, srv1207913814=6, srv850734492=93, srv1550351013=20, srv2106754936=59, srv1899855467=45, srv1233027431=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-14 03:54:06,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-14 03:54:06,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-14 03:54:06,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-14 03:54:06,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-14 03:54:06,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-14 03:54:06,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-14 03:54:06,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-14 03:54:06,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-14 03:54:06,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-14 03:54:06,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-14 03:54:06,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-14 03:54:06,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-14 03:54:06,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-14 03:54:06,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-14 03:54:06,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-14 03:54:06,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-14 03:54:06,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-14 03:54:06,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-14 03:54:06,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-14 03:54:06,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-14 03:54:06,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-14 03:54:06,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-14 03:54:06,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-14 03:54:06,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-14 03:54:06,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-14 03:54:06,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-14 03:54:06,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-14 03:54:06,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-14 03:54:06,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-14 03:54:06,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-14 03:54:06,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-14 03:54:06,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-14 03:54:06,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-14 03:54:06,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-14 03:54:06,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-14 03:54:06,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-14 03:54:06,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-14 03:54:06,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-14 03:54:06,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-14 03:54:06,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-14 03:54:06,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-14 03:54:06,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-14 03:54:06,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-14 03:54:06,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-14 03:54:06,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-14 03:54:06,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-14 03:54:06,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-14 03:54:06,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-14 03:54:06,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-14 03:54:06,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-14 03:54:06,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-14 03:54:06,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-14 03:54:06,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-14 03:54:06,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-14 03:54:06,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-14 03:54:06,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-14 03:54:06,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-14 03:54:06,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-14 03:54:06,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-14 03:54:06,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-14 03:54:06,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-14 03:54:06,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-14 03:54:06,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-14 03:54:06,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-14 03:54:06,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-14 03:54:06,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-14 03:54:06,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-14 03:54:06,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-14 03:54:06,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-14 03:54:06,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-14 03:54:06,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-14 03:54:06,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-14 03:54:06,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-14 03:54:06,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-14 03:54:06,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-14 03:54:06,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-14 03:54:06,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-14 03:54:06,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-14 03:54:06,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-14 03:54:06,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-14 03:54:06,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-14 03:54:06,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-14 03:54:06,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-14 03:54:06,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-14 03:54:06,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-14 03:54:06,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-14 03:54:06,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-14 03:54:06,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-14 03:54:06,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-14 03:54:06,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-14 03:54:06,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-14 03:54:06,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-14 03:54:06,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-14 03:54:06,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-14 03:54:06,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-14 03:54:06,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-14 03:54:06,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-14 03:54:06,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-14 03:54:06,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-14 03:54:06,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-14 03:54:06,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-14 03:54:06,137 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=6.744069053831948E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014982219165849825); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-14 03:54:06,137 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1139969612:83 , srv1693755428:83 , srv1711321672:83 , srv1829013479:83 , srv1988732331:83 , srv2127426464:83 , srv759501052:83 , srv850734492:83 , srv1020945953:84 , srv1107957691:84 , srv1128910468:84 , srv1173730115:84 , srv1195322497:84 , srv1207913814:84 , srv1233027431:84 , srv1279304942:84 , srv1291615432:84 , srv1306900847:84 , srv1349636155:84 , srv1349731607:84 , srv1386206261:84 , srv1432808463:84 , srv1458349237:84 , srv147499249:84 , srv1526155536:84 , srv1530954908:84 , srv1550351013:84 , srv1556464394:84 , srv1571911199:84 , srv1575276979:84 , srv1581898706:84 , srv1603872724:84 , srv1649930106:84 , srv1657988912:84 , srv1662487769:84 , srv1665917429:84 , srv1684390175:84 , srv1725704333:84 , srv1760773083:84 , srv1772095437:84 , srv1774599123:84 , srv1821646255:84 , srv1856679996:84 , srv1877264825:84 , srv1890761761:84 , srv1899855467:84 , srv191270933:84 , srv1929881319:84 , srv1936915723:84 , srv194163113:84 , srv1966700886:84 , srv2015110491:84 , srv2026940046:84 , srv2031597118:84 , srv2037000963:84 , srv2072769232:84 , srv2095036554:84 , srv2099139083:84 , srv2106754936:84 , srv211379109:84 , srv2114913159:84 , srv2136979912:84 , srv2142597021:84 , srv234590636:84 , srv242624596:84 , srv258610745:84 , srv30215481:84 , srv306757461:84 , srv306771103:84 , srv359556672:84 , srv39799323:84 , srv414323860:84 , srv43200949:84 , srv489576152:84 , srv578143334:84 , srv597426132:84 , srv611940201:84 , srv62158349:84 , srv629803089:84 , srv668142101:84 , srv683723546:84 , srv689738833:84 , srv73068430:84 , srv734708187:84 , srv768135144:84 , srv793428335:84 , srv803279479:84 , srv895631265:84 , srv946211317:84 , srv954825688:84 , srv985584639:84 , srv1529506007:85 , srv155884200:85 , srv1883622861:85 , srv1891942735:85 , srv215829334:85 , srv638224228:85 , srv864096522:85 , srv973676390:85 , srv771595062:684 }
2023-02-14 03:54:06,168 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=312 (was 305) - SystemLoadAverage LEAK? -, ProcessCount=171 (was 171), AvailableMemoryMB=3009 (was 3519)
2023-02-14 03:54:06,176 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=312, ProcessCount=171, AvailableMemoryMB=3009
2023-02-14 03:54:06,176 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-14 03:54:06,177 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-02-14 03:54:06,189 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1198077441:84 , srv1262684892:84 , srv1477174418:84 , srv1528082666:84 , srv1553200343:84 , srv1600879029:84 , srv1791379952:84 , srv1850534109:84 , srv193654546:84 , srv1958553645:84 , srv196399129:84 , srv1978976788:84 , srv2013999368:84 , srv2047302570:84 , srv2099208741:84 , srv277459261:84 , srv482780603:84 , srv508262330:84 , srv5492615:84 , srv558442462:84 , srv576446099:84 , srv596645413:84 , srv653972273:84 , srv67052868:84 , srv675161749:84 , srv704604766:84 , srv770303622:84 , srv952592389:84 , srv989804307:84 , srv167833127:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-02-14 03:54:06,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-14 03:54:06,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv596645413=22, srv1198077441=0, srv2099208741=15, srv1600879029=5, srv1978976788=12, srv1791379952=7, srv1850534109=8, srv67052868=24, srv952592389=28, srv1958553645=10, srv576446099=21, srv2013999368=13, srv1553200343=4, srv482780603=17, srv1528082666=3, srv277459261=16, srv5492615=19, srv508262330=18, srv558442462=20, srv1262684892=1, srv675161749=25, srv1477174418=2, srv196399129=11, srv989804307=29, srv704604766=26, srv193654546=9, srv2047302570=14, srv653972273=23, srv770303622=27, srv167833127=6} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-14 03:54:06,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-14 03:54:06,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-14 03:54:06,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-14 03:54:06,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-14 03:54:06,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-14 03:54:06,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-14 03:54:06,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-14 03:54:06,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-14 03:54:06,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-14 03:54:06,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-14 03:54:06,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-14 03:54:06,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-14 03:54:06,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-14 03:54:06,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-14 03:54:06,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-14 03:54:06,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-14 03:54:06,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-14 03:54:06,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-14 03:54:06,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-14 03:54:06,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-14 03:54:06,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-14 03:54:06,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-14 03:54:06,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-14 03:54:06,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-14 03:54:06,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-14 03:54:06,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-14 03:54:06,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-14 03:54:06,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-14 03:54:06,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-14 03:54:06,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-14 03:54:06,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-02-14 03:54:06,229 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.05250336317436, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-14 03:54:06,229 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912358479647251, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.05250336317436, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-02-14 03:56:06,210 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6365735 different iterations.  Found a solution that moves 1852 regions; Going from a computed imbalance of 0.9912358479647251 to a new imbalance of 4.322660390073086E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6859259259259259, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-14 03:56:06,216 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1198077441:90 , srv1262684892:90 , srv1477174418:90 , srv1528082666:90 , srv1553200343:90 , srv1600879029:90 , srv167833127:90 , srv1791379952:90 , srv1850534109:90 , srv193654546:90 , srv1958553645:90 , srv196399129:90 , srv1978976788:90 , srv2013999368:90 , srv2047302570:90 , srv2099208741:90 , srv277459261:90 , srv482780603:90 , srv508262330:90 , srv5492615:90 , srv558442462:90 , srv576446099:90 , srv596645413:90 , srv653972273:90 , srv67052868:90 , srv675161749:90 , srv704604766:90 , srv770303622:90 , srv952592389:90 , srv989804307:90 }
2023-02-14 03:56:06,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-14 03:56:06,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv596645413=22, srv1198077441=0, srv2099208741=15, srv1600879029=5, srv1978976788=12, srv1791379952=7, srv1850534109=8, srv67052868=24, srv952592389=28, srv1958553645=10, srv576446099=21, srv2013999368=13, srv1553200343=4, srv482780603=17, srv1528082666=3, srv277459261=16, srv5492615=19, srv508262330=18, srv558442462=20, srv1262684892=1, srv675161749=25, srv1477174418=2, srv196399129=11, srv989804307=29, srv704604766=26, srv193654546=9, srv2047302570=14, srv653972273=23, srv770303622=27, srv167833127=6} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-14 03:56:06,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-14 03:56:06,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-14 03:56:06,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-14 03:56:06,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-14 03:56:06,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-14 03:56:06,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-14 03:56:06,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-14 03:56:06,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-14 03:56:06,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-14 03:56:06,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-14 03:56:06,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-14 03:56:06,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-14 03:56:06,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-14 03:56:06,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-14 03:56:06,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-14 03:56:06,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-14 03:56:06,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-14 03:56:06,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-14 03:56:06,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-14 03:56:06,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-14 03:56:06,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-14 03:56:06,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-14 03:56:06,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-14 03:56:06,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-14 03:56:06,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-14 03:56:06,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-14 03:56:06,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-14 03:56:06,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-14 03:56:06,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-14 03:56:06,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-14 03:56:06,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-02-14 03:56:06,234 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-14 03:56:06,234 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1198077441:90 , srv1262684892:90 , srv1477174418:90 , srv1528082666:90 , srv1553200343:90 , srv1600879029:90 , srv167833127:90 , srv1791379952:90 , srv1850534109:90 , srv193654546:90 , srv1958553645:90 , srv196399129:90 , srv1978976788:90 , srv2013999368:90 , srv2047302570:90 , srv2099208741:90 , srv277459261:90 , srv482780603:90 , srv508262330:90 , srv5492615:90 , srv558442462:90 , srv576446099:90 , srv596645413:90 , srv653972273:90 , srv67052868:90 , srv675161749:90 , srv704604766:90 , srv770303622:90 , srv952592389:90 , srv989804307:90 }
2023-02-14 03:56:06,246 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=164 (was 312), ProcessCount=170 (was 171), AvailableMemoryMB=2828 (was 3009)