2023-04-11 20:03:56,411 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-04-11 20:03:57,269 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-04-11 20:03:57,385 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-04-11 20:03:57,523 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-11 20:03:57,546 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-04-11 20:03:57,592 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=861, ProcessCount=166, AvailableMemoryMB=5720
2023-04-11 20:03:57,594 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-11 20:03:57,595 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-04-11 20:03:58,075 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1002788996:84 , srv1006649448:84 , srv1019592851:84 , srv1040958416:84 , srv105407453:84 , srv1074760460:84 , srv1079019892:84 , srv1080597681:84 , srv1080715976:84 , srv1123530858:84 , srv1146770752:84 , srv1162362497:84 , srv124007561:84 , srv1290362794:84 , srv1292670493:84 , srv1299365920:84 , srv1304563804:84 , srv1359802974:84 , srv1362480562:84 , srv1380686652:84 , srv1402245096:84 , srv1405459484:84 , srv1405768853:84 , srv1417214683:84 , srv1417295236:84 , srv1418166188:84 , srv1425518296:84 , srv1496026944:84 , srv1527347704:84 , srv1553447339:84 , srv1574189060:84 , srv1579095824:84 , srv1654328138:84 , srv1680153861:84 , srv1680959874:84 , srv168414045:84 , srv1686088543:84 , srv1705160139:84 , srv1705544566:84 , srv1755996572:84 , srv1757387353:84 , srv1757464517:84 , srv1791939550:84 , srv1864184030:84 , srv1898126986:84 , srv1917333517:84 , srv1920549851:84 , srv1924756842:84 , srv1935849729:84 , srv1944862683:84 , srv1967615879:84 , srv2021797803:84 , srv2035951106:84 , srv2037271645:84 , srv2044352453:84 , srv2072151200:84 , srv2089823753:84 , srv2113145838:84 , srv220751828:84 , srv226112327:84 , srv237329048:84 , srv257062274:84 , srv263004393:84 , srv32955242:84 , srv401974939:84 , srv415488731:84 , srv439677912:84 , srv457722908:84 , srv461085933:84 , srv47982392:84 , srv482493942:84 , srv503513922:84 , srv505518009:84 , srv507931530:84 , srv518931405:84 , srv567550137:84 , srv620021480:84 , srv621715005:84 , srv62415014:84 , srv661363550:84 , srv687668001:84 , srv726554996:84 , srv731385793:84 , srv740354971:84 , srv741481281:84 , srv790266314:84 , srv826706471:84 , srv847884065:84 , srv863982248:84 , srv865034866:84 , srv887414408:84 , srv890341323:84 , srv938234584:84 , srv940466384:84 , srv941335166:84 , srv959288445:84 , srv977213825:84 , srv996955990:84 , srv999859533:84 , srv1627274012:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-04-11 20:03:58,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-11 20:03:58,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv507931530=74, srv457722908=68, srv1290362794=13, srv124007561=12, srv226112327=60, srv1405459484=21, srv1920549851=47, srv439677912=67, srv1579095824=31, srv865034866=90, srv1755996572=40, srv32955242=64, srv237329048=61, srv1079019892=6, srv263004393=63, srv401974939=65, srv1304563804=16, srv740354971=84, srv1680959874=35, srv1292670493=14, srv790266314=86, srv1917333517=46, srv2021797803=52, srv687668001=81, srv731385793=83, srv826706471=87, srv1006649448=1, srv47982392=70, srv62415014=79, srv726554996=82, srv2037271645=54, srv503513922=72, srv2113145838=58, srv1359802974=17, srv1002788996=0, srv1080597681=7, srv1362480562=18, srv1864184030=44, srv661363550=80, srv1757464517=42, srv1299365920=15, srv415488731=66, srv621715005=78, srv1040958416=3, srv890341323=92, srv1627274012=32, srv977213825=97, srv257062274=62, srv999859533=99, srv2072151200=56, srv482493942=71, srv959288445=96, srv1935849729=49, srv1123530858=9, srv1418166188=25, srv1654328138=33, srv1380686652=19, srv1425518296=26, srv461085933=69, srv168414045=36, srv2044352453=55, srv1527347704=28, srv1705544566=39, srv1757387353=41, srv1680153861=34, srv1417214683=23, srv1405768853=22, srv220751828=59, srv1574189060=30, srv1898126986=45, srv1791939550=43, srv2035951106=53, srv1074760460=5, srv863982248=89, srv1019592851=2, srv2089823753=57, srv505518009=73, srv518931405=75, srv1162362497=11, srv1146770752=10, srv1944862683=50, srv741481281=85, srv887414408=91, srv1686088543=37, srv1402245096=20, srv1705160139=38, srv996955990=98, srv105407453=4, srv941335166=95, srv938234584=93, srv940466384=94, srv620021480=77, srv1496026944=27, srv1553447339=29, srv1417295236=24, srv1080715976=8, srv1967615879=51, srv1924756842=48, srv567550137=76, srv847884065=88} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-11 20:03:58,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-11 20:03:58,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-11 20:03:58,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-11 20:03:58,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-11 20:03:58,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-11 20:03:58,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-11 20:03:58,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-11 20:03:58,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-11 20:03:58,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-11 20:03:58,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-11 20:03:58,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-11 20:03:58,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-11 20:03:58,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-11 20:03:58,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-11 20:03:58,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-11 20:03:58,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-11 20:03:58,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-11 20:03:58,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-11 20:03:58,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-11 20:03:58,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-11 20:03:58,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-11 20:03:58,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-11 20:03:58,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-11 20:03:58,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-11 20:03:58,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-11 20:03:58,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-11 20:03:58,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-11 20:03:58,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-11 20:03:58,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-11 20:03:58,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-11 20:03:58,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-11 20:03:58,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-11 20:03:58,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-11 20:03:58,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-11 20:03:58,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-11 20:03:58,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-11 20:03:58,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-11 20:03:58,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-11 20:03:58,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-11 20:03:58,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-11 20:03:58,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-11 20:03:58,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-11 20:03:58,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-11 20:03:58,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-11 20:03:58,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-11 20:03:58,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-11 20:03:58,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-11 20:03:58,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-11 20:03:58,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-11 20:03:58,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-11 20:03:58,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-11 20:03:58,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-11 20:03:58,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-11 20:03:58,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-11 20:03:58,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-11 20:03:58,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-11 20:03:58,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-11 20:03:58,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-11 20:03:58,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-11 20:03:58,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-11 20:03:58,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-11 20:03:58,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-11 20:03:58,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-11 20:03:58,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-11 20:03:58,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-11 20:03:58,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-11 20:03:58,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-11 20:03:58,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-11 20:03:58,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-11 20:03:58,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-11 20:03:58,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-11 20:03:58,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-11 20:03:58,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-11 20:03:58,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-11 20:03:58,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-11 20:03:58,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-11 20:03:58,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-11 20:03:58,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-11 20:03:58,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-11 20:03:58,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-11 20:03:58,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-11 20:03:58,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-11 20:03:58,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-11 20:03:58,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-11 20:03:58,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-11 20:03:58,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-11 20:03:58,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-11 20:03:58,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-11 20:03:58,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-11 20:03:58,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-11 20:03:58,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-11 20:03:58,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-11 20:03:58,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-11 20:03:58,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-11 20:03:58,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-11 20:03:58,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-11 20:03:58,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-11 20:03:58,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-11 20:03:58,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-11 20:03:58,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-11 20:03:58,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-11 20:03:59,028 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-11 20:03:59,029 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-11 20:03:59,029 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-11 20:03:59,618 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 1209 ms to try 5000 different iterations.  Found a solution that moves 4651 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1839369713910497. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03902988709047272, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.060003740998226654, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5167777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03902988709047272, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.179, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24766666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-11 20:03:59,658 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1579095824:80 , srv1080715976:82 , srv1123530858:82 , srv1496026944:82 , srv1418166188:83 , srv2035951106:83 , srv726554996:83 , srv887414408:83 , srv124007561:84 , srv1290362794:84 , srv1405459484:84 , srv1425518296:84 , srv1527347704:84 , srv1705160139:84 , srv1935849729:84 , srv518931405:84 , srv790266314:84 , srv890341323:84 , srv938234584:84 , srv1019592851:85 , srv1146770752:85 , srv1380686652:85 , srv1417295236:85 , srv1574189060:85 , srv1680153861:85 , srv1791939550:85 , srv2072151200:85 , srv62415014:85 , srv863982248:85 , srv999859533:85 , srv105407453:86 , srv1074760460:86 , srv1080597681:86 , srv1299365920:86 , srv1304563804:86 , srv1359802974:86 , srv1405768853:86 , srv1553447339:86 , srv1917333517:86 , srv237329048:86 , srv457722908:86 , srv621715005:86 , srv741481281:86 , srv941335166:86 , srv959288445:86 , srv977213825:86 , srv996955990:86 , srv1162362497:87 , srv1292670493:87 , srv1417214683:87 , srv1680959874:87 , srv1686088543:87 , srv1705544566:87 , srv1757387353:87 , srv1864184030:87 , srv1898126986:87 , srv1920549851:87 , srv1924756842:87 , srv2021797803:87 , srv2037271645:87 , srv2044352453:87 , srv220751828:87 , srv257062274:87 , srv415488731:87 , srv439677912:87 , srv461085933:87 , srv47982392:87 , srv731385793:87 , srv826706471:87 , srv847884065:87 , srv940466384:87 , srv1002788996:88 , srv1006649448:88 , srv1040958416:88 , srv1362480562:88 , srv1654328138:88 , srv168414045:88 , srv1757464517:88 , srv503513922:88 , srv505518009:88 , srv687668001:88 , srv865034866:88 , srv1079019892:89 , srv1755996572:89 , srv1944862683:89 , srv1967615879:89 , srv2089823753:89 , srv2113145838:89 , srv263004393:89 , srv401974939:89 , srv482493942:89 , srv620021480:89 , srv661363550:89 , srv740354971:89 , srv226112327:90 , srv32955242:90 , srv507931530:92 , srv567550137:92 , srv1402245096:96 , srv1627274012:437 }
2023-04-11 20:03:59,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-11 20:03:59,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv507931530=74, srv457722908=68, srv1290362794=13, srv124007561=12, srv226112327=60, srv1405459484=21, srv1920549851=47, srv439677912=67, srv1579095824=31, srv865034866=90, srv1755996572=40, srv32955242=64, srv237329048=61, srv1079019892=6, srv263004393=63, srv401974939=65, srv1304563804=16, srv740354971=84, srv1680959874=35, srv1292670493=14, srv790266314=86, srv1917333517=46, srv2021797803=52, srv687668001=81, srv731385793=83, srv826706471=87, srv1006649448=1, srv47982392=70, srv62415014=79, srv726554996=82, srv2037271645=54, srv503513922=72, srv2113145838=58, srv1359802974=17, srv1002788996=0, srv1080597681=7, srv1362480562=18, srv1864184030=44, srv661363550=80, srv1757464517=42, srv1299365920=15, srv415488731=66, srv621715005=78, srv1040958416=3, srv890341323=92, srv1627274012=32, srv977213825=97, srv257062274=62, srv999859533=99, srv2072151200=56, srv482493942=71, srv959288445=96, srv1935849729=49, srv1123530858=9, srv1418166188=25, srv1654328138=33, srv1380686652=19, srv1425518296=26, srv461085933=69, srv168414045=36, srv2044352453=55, srv1527347704=28, srv1705544566=39, srv1757387353=41, srv1680153861=34, srv1417214683=23, srv1405768853=22, srv220751828=59, srv1574189060=30, srv1898126986=45, srv1791939550=43, srv2035951106=53, srv1074760460=5, srv863982248=89, srv1019592851=2, srv2089823753=57, srv505518009=73, srv518931405=75, srv1162362497=11, srv1146770752=10, srv1944862683=50, srv741481281=85, srv887414408=91, srv1686088543=37, srv1402245096=20, srv1705160139=38, srv996955990=98, srv105407453=4, srv941335166=95, srv938234584=93, srv940466384=94, srv620021480=77, srv1496026944=27, srv1553447339=29, srv1417295236=24, srv1080715976=8, srv1967615879=51, srv1924756842=48, srv567550137=76, srv847884065=88} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-11 20:03:59,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-11 20:03:59,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-11 20:03:59,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-11 20:03:59,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-11 20:03:59,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-11 20:03:59,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-11 20:03:59,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-11 20:03:59,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-11 20:03:59,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-11 20:03:59,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-11 20:03:59,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-11 20:03:59,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-11 20:03:59,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-11 20:03:59,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-11 20:03:59,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-11 20:03:59,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-11 20:03:59,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-11 20:03:59,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-11 20:03:59,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-11 20:03:59,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-11 20:03:59,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-11 20:03:59,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-11 20:03:59,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-11 20:03:59,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-11 20:03:59,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-11 20:03:59,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-11 20:03:59,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-11 20:03:59,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-11 20:03:59,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-11 20:03:59,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-11 20:03:59,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-11 20:03:59,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-11 20:03:59,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-11 20:03:59,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-11 20:03:59,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-11 20:03:59,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-11 20:03:59,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-11 20:03:59,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-11 20:03:59,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-11 20:03:59,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-11 20:03:59,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-11 20:03:59,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-11 20:03:59,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-11 20:03:59,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-11 20:03:59,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-11 20:03:59,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-11 20:03:59,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-11 20:03:59,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-11 20:03:59,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-11 20:03:59,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-11 20:03:59,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-11 20:03:59,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-11 20:03:59,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-11 20:03:59,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-11 20:03:59,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-11 20:03:59,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-11 20:03:59,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-11 20:03:59,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-11 20:03:59,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-11 20:03:59,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-11 20:03:59,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-11 20:03:59,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-11 20:03:59,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-11 20:03:59,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-11 20:03:59,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-11 20:03:59,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-11 20:03:59,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-11 20:03:59,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-11 20:03:59,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-11 20:03:59,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-11 20:03:59,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-11 20:03:59,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-11 20:03:59,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-11 20:03:59,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-11 20:03:59,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-11 20:03:59,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-11 20:03:59,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-11 20:03:59,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-11 20:03:59,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-11 20:03:59,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-11 20:03:59,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-11 20:03:59,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-11 20:03:59,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-11 20:03:59,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-11 20:03:59,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-11 20:03:59,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-11 20:03:59,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-11 20:03:59,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-11 20:03:59,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-11 20:03:59,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-11 20:03:59,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-11 20:03:59,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-11 20:03:59,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-11 20:03:59,717 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-11 20:03:59,717 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-11 20:03:59,717 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-11 20:03:59,717 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-11 20:03:59,717 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-11 20:03:59,717 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-11 20:03:59,717 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-11 20:03:59,717 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-11 20:03:59,887 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03902988709047272, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.060003740998226654, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03902988709047272, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.179, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24766666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-11 20:03:59,887 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-11 20:03:59,887 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.1839044043929813, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03902988709047272, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.060003740998226654, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03902988709047272, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.179, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24766666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-11 20:04:00,220 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 473 ms to try 5000 different iterations.  Found a solution that moves 2724 regions; Going from a computed imbalance of 0.1839044043929813 to a new imbalance of 0.014125313519429663. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020827140867933833); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03480585886668811, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.30266666666666664, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020827140867933833); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010833333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0455, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-11 20:04:00,245 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1402245096:75 , srv507931530:80 , srv1002788996:81 , srv32955242:81 , srv1079019892:82 , srv2089823753:82 , srv2113145838:82 , srv482493942:82 , srv567550137:82 , srv1006649448:83 , srv1686088543:83 , srv1864184030:83 , srv1898126986:83 , srv401974939:83 , srv461085933:83 , srv740354971:83 , srv1292670493:84 , srv1405768853:84 , srv1654328138:84 , srv1755996572:84 , srv1757387353:84 , srv1917333517:84 , srv1924756842:84 , srv1944862683:84 , srv220751828:84 , srv257062274:84 , srv503513922:84 , srv661363550:84 , srv1040958416:85 , srv1080597681:85 , srv1299365920:85 , srv1359802974:85 , srv1362480562:85 , srv1417214683:85 , srv1418166188:85 , srv1553447339:85 , srv168414045:85 , srv1757464517:85 , srv1920549851:85 , srv1967615879:85 , srv2037271645:85 , srv226112327:85 , srv415488731:85 , srv620021480:85 , srv621715005:85 , srv687668001:85 , srv731385793:85 , srv826706471:85 , srv863982248:85 , srv890341323:85 , srv959288445:85 , srv996955990:85 , srv1019592851:86 , srv105407453:86 , srv1074760460:86 , srv1304563804:86 , srv2021797803:86 , srv237329048:86 , srv263004393:86 , srv439677912:86 , srv457722908:86 , srv47982392:86 , srv505518009:86 , srv518931405:86 , srv847884065:86 , srv865034866:86 , srv977213825:86 , srv1162362497:87 , srv1380686652:87 , srv1680153861:87 , srv1705544566:87 , srv1791939550:87 , srv2072151200:87 , srv62415014:87 , srv938234584:87 , srv940466384:87 , srv1290362794:88 , srv1417295236:88 , srv1496026944:88 , srv1680959874:88 , srv2044352453:88 , srv941335166:88 , srv999859533:88 , srv1080715976:89 , srv1123530858:89 , srv1146770752:89 , srv124007561:89 , srv1405459484:89 , srv1425518296:89 , srv1574189060:89 , srv1935849729:89 , srv726554996:89 , srv741481281:89 , srv790266314:89 , srv1527347704:90 , srv1705160139:90 , srv887414408:90 , srv2035951106:91 , srv1579095824:93 , srv1627274012:522 }
2023-04-11 20:04:00,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-11 20:04:00,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv507931530=74, srv457722908=68, srv1290362794=13, srv124007561=12, srv226112327=60, srv1405459484=21, srv1920549851=47, srv439677912=67, srv1579095824=31, srv865034866=90, srv1755996572=40, srv32955242=64, srv237329048=61, srv1079019892=6, srv263004393=63, srv401974939=65, srv1304563804=16, srv740354971=84, srv1680959874=35, srv1292670493=14, srv790266314=86, srv1917333517=46, srv2021797803=52, srv687668001=81, srv731385793=83, srv826706471=87, srv1006649448=1, srv47982392=70, srv62415014=79, srv726554996=82, srv2037271645=54, srv503513922=72, srv2113145838=58, srv1359802974=17, srv1002788996=0, srv1080597681=7, srv1362480562=18, srv1864184030=44, srv661363550=80, srv1757464517=42, srv1299365920=15, srv415488731=66, srv621715005=78, srv1040958416=3, srv890341323=92, srv1627274012=32, srv977213825=97, srv257062274=62, srv999859533=99, srv2072151200=56, srv482493942=71, srv959288445=96, srv1935849729=49, srv1123530858=9, srv1418166188=25, srv1654328138=33, srv1380686652=19, srv1425518296=26, srv461085933=69, srv168414045=36, srv2044352453=55, srv1527347704=28, srv1705544566=39, srv1757387353=41, srv1680153861=34, srv1417214683=23, srv1405768853=22, srv220751828=59, srv1574189060=30, srv1898126986=45, srv1791939550=43, srv2035951106=53, srv1074760460=5, srv863982248=89, srv1019592851=2, srv2089823753=57, srv505518009=73, srv518931405=75, srv1162362497=11, srv1146770752=10, srv1944862683=50, srv741481281=85, srv887414408=91, srv1686088543=37, srv1402245096=20, srv1705160139=38, srv996955990=98, srv105407453=4, srv941335166=95, srv938234584=93, srv940466384=94, srv620021480=77, srv1496026944=27, srv1553447339=29, srv1417295236=24, srv1080715976=8, srv1967615879=51, srv1924756842=48, srv567550137=76, srv847884065=88} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-11 20:04:00,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-11 20:04:00,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-11 20:04:00,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-11 20:04:00,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-11 20:04:00,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-11 20:04:00,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-11 20:04:00,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-11 20:04:00,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-11 20:04:00,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-11 20:04:00,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-11 20:04:00,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-11 20:04:00,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-11 20:04:00,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-11 20:04:00,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-11 20:04:00,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-11 20:04:00,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-11 20:04:00,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-11 20:04:00,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-11 20:04:00,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-11 20:04:00,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-11 20:04:00,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-11 20:04:00,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-11 20:04:00,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-11 20:04:00,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-11 20:04:00,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-11 20:04:00,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-11 20:04:00,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-11 20:04:00,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-11 20:04:00,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-11 20:04:00,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-11 20:04:00,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-11 20:04:00,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-11 20:04:00,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-11 20:04:00,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-11 20:04:00,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-11 20:04:00,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-11 20:04:00,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-11 20:04:00,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-11 20:04:00,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-11 20:04:00,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-11 20:04:00,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-11 20:04:00,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-11 20:04:00,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-11 20:04:00,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-11 20:04:00,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-11 20:04:00,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-11 20:04:00,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-11 20:04:00,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-11 20:04:00,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-11 20:04:00,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-11 20:04:00,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-11 20:04:00,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-11 20:04:00,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-11 20:04:00,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-11 20:04:00,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-11 20:04:00,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-11 20:04:00,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-11 20:04:00,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-11 20:04:00,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-11 20:04:00,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-11 20:04:00,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-11 20:04:00,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-11 20:04:00,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-11 20:04:00,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-11 20:04:00,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-11 20:04:00,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-11 20:04:00,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-11 20:04:00,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-11 20:04:00,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-11 20:04:00,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-11 20:04:00,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-11 20:04:00,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-11 20:04:00,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-11 20:04:00,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-11 20:04:00,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-11 20:04:00,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-11 20:04:00,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-11 20:04:00,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-11 20:04:00,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-11 20:04:00,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-11 20:04:00,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-11 20:04:00,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-11 20:04:00,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-11 20:04:00,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-11 20:04:00,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-11 20:04:00,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-11 20:04:00,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-11 20:04:00,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-11 20:04:00,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-11 20:04:00,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-11 20:04:00,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-11 20:04:00,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-11 20:04:00,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-11 20:04:00,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-11 20:04:00,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-11 20:04:00,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-11 20:04:00,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-11 20:04:00,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-11 20:04:00,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-11 20:04:00,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-11 20:04:00,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-11 20:04:00,538 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020827140867933833); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03480585886668811, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020827140867933833); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010833333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0455, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-11 20:04:00,538 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-11 20:04:00,539 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.01410623966375597, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020827140867933833); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03480585886668811, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020827140867933833); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010833333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0455, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-11 20:04:00,987 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 591 ms to try 5000 different iterations.  Found a solution that moves 1025 regions; Going from a computed imbalance of 0.01410623966375597 to a new imbalance of 8.567198369870338E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005329871678683814); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008026331525381385); MoveCostFunction : (multiplier=7.0, imbalance=0.11388888888888889, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005329871678683814); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004583333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-11 20:04:00,989 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv263004393:82 , srv1146770752:83 , srv1680959874:83 , srv1705160139:83 , srv1967615879:83 , srv2044352453:83 , srv226112327:83 , srv505518009:83 , srv741481281:83 , srv790266314:83 , srv865034866:83 , srv1040958416:84 , srv1380686652:84 , srv1425518296:84 , srv1574189060:84 , srv168414045:84 , srv1935849729:84 , srv1944862683:84 , srv2035951106:84 , srv47982392:84 , srv567550137:84 , srv661363550:84 , srv940466384:84 , srv941335166:84 , srv1074760460:85 , srv1079019892:85 , srv1162362497:85 , srv124007561:85 , srv1290362794:85 , srv1299365920:85 , srv1362480562:85 , srv1405459484:85 , srv1417295236:85 , srv1527347704:85 , srv1579095824:85 , srv1755996572:85 , srv1757464517:85 , srv1791939550:85 , srv2037271645:85 , srv2072151200:85 , srv415488731:85 , srv439677912:85 , srv457722908:85 , srv620021480:85 , srv62415014:85 , srv687668001:85 , srv826706471:85 , srv847884065:85 , srv887414408:85 , srv977213825:85 , srv999859533:85 , srv1006649448:86 , srv1019592851:86 , srv105407453:86 , srv1080597681:86 , srv1080715976:86 , srv1292670493:86 , srv1304563804:86 , srv1402245096:86 , srv1417214683:86 , srv1553447339:86 , srv1654328138:86 , srv1680153861:86 , srv1705544566:86 , srv1757387353:86 , srv1920549851:86 , srv1924756842:86 , srv2021797803:86 , srv2089823753:86 , srv2113145838:86 , srv220751828:86 , srv237329048:86 , srv401974939:86 , srv503513922:86 , srv507931530:86 , srv726554996:86 , srv731385793:86 , srv740354971:86 , srv938234584:86 , srv959288445:86 , srv996955990:86 , srv1123530858:87 , srv1359802974:87 , srv1496026944:87 , srv1686088543:87 , srv1864184030:87 , srv1898126986:87 , srv257062274:87 , srv32955242:87 , srv482493942:87 , srv518931405:87 , srv621715005:87 , srv863982248:87 , srv1405768853:88 , srv1917333517:88 , srv461085933:88 , srv890341323:88 , srv1002788996:89 , srv1418166188:90 , srv1627274012:546 }
2023-04-11 20:04:00,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-11 20:04:00,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv507931530=74, srv457722908=68, srv1290362794=13, srv124007561=12, srv226112327=60, srv1405459484=21, srv1920549851=47, srv439677912=67, srv1579095824=31, srv865034866=90, srv1755996572=40, srv32955242=64, srv237329048=61, srv1079019892=6, srv263004393=63, srv401974939=65, srv1304563804=16, srv740354971=84, srv1680959874=35, srv1292670493=14, srv790266314=86, srv1917333517=46, srv2021797803=52, srv687668001=81, srv731385793=83, srv826706471=87, srv1006649448=1, srv47982392=70, srv62415014=79, srv726554996=82, srv2037271645=54, srv503513922=72, srv2113145838=58, srv1359802974=17, srv1002788996=0, srv1080597681=7, srv1362480562=18, srv1864184030=44, srv661363550=80, srv1757464517=42, srv1299365920=15, srv415488731=66, srv621715005=78, srv1040958416=3, srv890341323=92, srv1627274012=32, srv977213825=97, srv257062274=62, srv999859533=99, srv2072151200=56, srv482493942=71, srv959288445=96, srv1935849729=49, srv1123530858=9, srv1418166188=25, srv1654328138=33, srv1380686652=19, srv1425518296=26, srv461085933=69, srv168414045=36, srv2044352453=55, srv1527347704=28, srv1705544566=39, srv1757387353=41, srv1680153861=34, srv1417214683=23, srv1405768853=22, srv220751828=59, srv1574189060=30, srv1898126986=45, srv1791939550=43, srv2035951106=53, srv1074760460=5, srv863982248=89, srv1019592851=2, srv2089823753=57, srv505518009=73, srv518931405=75, srv1162362497=11, srv1146770752=10, srv1944862683=50, srv741481281=85, srv887414408=91, srv1686088543=37, srv1402245096=20, srv1705160139=38, srv996955990=98, srv105407453=4, srv941335166=95, srv938234584=93, srv940466384=94, srv620021480=77, srv1496026944=27, srv1553447339=29, srv1417295236=24, srv1080715976=8, srv1967615879=51, srv1924756842=48, srv567550137=76, srv847884065=88} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-11 20:04:01,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-11 20:04:01,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-11 20:04:01,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-11 20:04:01,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-11 20:04:01,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-11 20:04:01,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-11 20:04:01,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-11 20:04:01,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-11 20:04:01,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-11 20:04:01,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-11 20:04:01,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-11 20:04:01,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-11 20:04:01,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-11 20:04:01,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-11 20:04:01,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-11 20:04:01,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-11 20:04:01,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-11 20:04:01,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-11 20:04:01,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-11 20:04:01,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-11 20:04:01,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-11 20:04:01,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-11 20:04:01,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-11 20:04:01,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-11 20:04:01,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-11 20:04:01,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-11 20:04:01,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-11 20:04:01,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-11 20:04:01,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-11 20:04:01,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-11 20:04:01,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-11 20:04:01,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-11 20:04:01,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-11 20:04:01,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-11 20:04:01,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-11 20:04:01,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-11 20:04:01,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-11 20:04:01,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-11 20:04:01,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-11 20:04:01,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-11 20:04:01,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-11 20:04:01,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-11 20:04:01,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-11 20:04:01,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-11 20:04:01,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-11 20:04:01,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-11 20:04:01,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-11 20:04:01,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-11 20:04:01,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-11 20:04:01,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-11 20:04:01,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-11 20:04:01,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-11 20:04:01,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-11 20:04:01,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-11 20:04:01,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-11 20:04:01,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-11 20:04:01,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-11 20:04:01,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-11 20:04:01,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-11 20:04:01,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-11 20:04:01,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-11 20:04:01,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-11 20:04:01,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-11 20:04:01,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-11 20:04:01,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-11 20:04:01,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-11 20:04:01,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-11 20:04:01,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-11 20:04:01,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-11 20:04:01,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-11 20:04:01,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-11 20:04:01,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-11 20:04:01,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-11 20:04:01,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-11 20:04:01,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-11 20:04:01,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-11 20:04:01,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-11 20:04:01,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-11 20:04:01,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-11 20:04:01,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-11 20:04:01,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-11 20:04:01,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-11 20:04:01,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-11 20:04:01,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-11 20:04:01,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-11 20:04:01,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-11 20:04:01,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-11 20:04:01,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-11 20:04:01,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-11 20:04:01,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-11 20:04:01,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-11 20:04:01,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-11 20:04:01,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-11 20:04:01,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-11 20:04:01,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-11 20:04:01,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-11 20:04:01,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-11 20:04:01,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-11 20:04:01,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-11 20:04:01,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-11 20:04:01,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-11 20:04:01,279 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005329871678683814); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008026331525381385); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005329871678683814); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004583333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-11 20:04:01,279 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-11 20:04:01,280 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=8.495426335855896E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005329871678683814); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008026331525381385); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005329871678683814); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004583333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-11 20:04:01,767 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 698 ms to try 5000 different iterations.  Found a solution that moves 300 regions; Going from a computed imbalance of 8.495426335855896E-4 to a new imbalance of 3.713341414437434E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.23782800878924E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0022222222222222222); MoveCostFunction : (multiplier=7.0, imbalance=0.03333333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.23782800878924E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-11 20:04:01,768 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv105407453:83 , srv1162362497:83 , srv1405459484:83 , srv1527347704:83 , srv1705544566:83 , srv503513922:83 , srv620021480:83 , srv740354971:83 , srv1002788996:84 , srv1080715976:84 , srv1123530858:84 , srv124007561:84 , srv1304563804:84 , srv1359802974:84 , srv1362480562:84 , srv1402245096:84 , srv1405768853:84 , srv1417214683:84 , srv1417295236:84 , srv1418166188:84 , srv1574189060:84 , srv1579095824:84 , srv1680153861:84 , srv1680959874:84 , srv1686088543:84 , srv1755996572:84 , srv1757464517:84 , srv1917333517:84 , srv1920549851:84 , srv1924756842:84 , srv2021797803:84 , srv2035951106:84 , srv2044352453:84 , srv226112327:84 , srv237329048:84 , srv257062274:84 , srv32955242:84 , srv401974939:84 , srv439677912:84 , srv457722908:84 , srv461085933:84 , srv482493942:84 , srv567550137:84 , srv621715005:84 , srv62415014:84 , srv687668001:84 , srv726554996:84 , srv731385793:84 , srv847884065:84 , srv887414408:84 , srv890341323:84 , srv941335166:84 , srv999859533:84 , srv1006649448:85 , srv1019592851:85 , srv1040958416:85 , srv1080597681:85 , srv1146770752:85 , srv1292670493:85 , srv1425518296:85 , srv1496026944:85 , srv1654328138:85 , srv168414045:85 , srv1705160139:85 , srv1757387353:85 , srv1791939550:85 , srv1864184030:85 , srv1898126986:85 , srv1935849729:85 , srv1944862683:85 , srv2037271645:85 , srv2072151200:85 , srv2089823753:85 , srv2113145838:85 , srv220751828:85 , srv263004393:85 , srv415488731:85 , srv47982392:85 , srv505518009:85 , srv507931530:85 , srv518931405:85 , srv661363550:85 , srv741481281:85 , srv826706471:85 , srv865034866:85 , srv938234584:85 , srv940466384:85 , srv959288445:85 , srv977213825:85 , srv996955990:85 , srv1074760460:86 , srv1079019892:86 , srv1290362794:86 , srv1299365920:86 , srv1380686652:86 , srv1553447339:86 , srv1967615879:86 , srv790266314:86 , srv863982248:86 , srv1627274012:637 }
2023-04-11 20:04:01,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-11 20:04:01,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv507931530=74, srv457722908=68, srv1290362794=13, srv124007561=12, srv226112327=60, srv1405459484=21, srv1920549851=47, srv439677912=67, srv1579095824=31, srv865034866=90, srv1755996572=40, srv32955242=64, srv237329048=61, srv1079019892=6, srv263004393=63, srv401974939=65, srv1304563804=16, srv740354971=84, srv1680959874=35, srv1292670493=14, srv790266314=86, srv1917333517=46, srv2021797803=52, srv687668001=81, srv731385793=83, srv826706471=87, srv1006649448=1, srv47982392=70, srv62415014=79, srv726554996=82, srv2037271645=54, srv503513922=72, srv2113145838=58, srv1359802974=17, srv1002788996=0, srv1080597681=7, srv1362480562=18, srv1864184030=44, srv661363550=80, srv1757464517=42, srv1299365920=15, srv415488731=66, srv621715005=78, srv1040958416=3, srv890341323=92, srv1627274012=32, srv977213825=97, srv257062274=62, srv999859533=99, srv2072151200=56, srv482493942=71, srv959288445=96, srv1935849729=49, srv1123530858=9, srv1418166188=25, srv1654328138=33, srv1380686652=19, srv1425518296=26, srv461085933=69, srv168414045=36, srv2044352453=55, srv1527347704=28, srv1705544566=39, srv1757387353=41, srv1680153861=34, srv1417214683=23, srv1405768853=22, srv220751828=59, srv1574189060=30, srv1898126986=45, srv1791939550=43, srv2035951106=53, srv1074760460=5, srv863982248=89, srv1019592851=2, srv2089823753=57, srv505518009=73, srv518931405=75, srv1162362497=11, srv1146770752=10, srv1944862683=50, srv741481281=85, srv887414408=91, srv1686088543=37, srv1402245096=20, srv1705160139=38, srv996955990=98, srv105407453=4, srv941335166=95, srv938234584=93, srv940466384=94, srv620021480=77, srv1496026944=27, srv1553447339=29, srv1417295236=24, srv1080715976=8, srv1967615879=51, srv1924756842=48, srv567550137=76, srv847884065=88} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-11 20:04:01,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-11 20:04:01,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-11 20:04:01,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-11 20:04:01,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-11 20:04:01,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-11 20:04:01,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-11 20:04:01,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-11 20:04:01,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-11 20:04:01,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-11 20:04:01,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-11 20:04:01,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-11 20:04:01,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-11 20:04:01,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-11 20:04:01,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-11 20:04:01,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-11 20:04:01,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-11 20:04:01,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-11 20:04:01,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-11 20:04:01,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-11 20:04:01,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-11 20:04:01,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-11 20:04:01,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-11 20:04:01,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-11 20:04:01,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-11 20:04:01,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-11 20:04:01,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-11 20:04:01,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-11 20:04:01,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-11 20:04:01,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-11 20:04:01,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-11 20:04:01,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-11 20:04:01,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-11 20:04:01,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-11 20:04:01,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-11 20:04:01,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-11 20:04:01,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-11 20:04:01,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-11 20:04:01,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-11 20:04:01,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-11 20:04:01,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-11 20:04:01,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-11 20:04:01,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-11 20:04:01,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-11 20:04:01,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-11 20:04:01,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-11 20:04:01,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-11 20:04:01,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-11 20:04:01,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-11 20:04:01,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-11 20:04:01,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-11 20:04:01,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-11 20:04:01,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-11 20:04:01,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-11 20:04:01,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-11 20:04:01,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-11 20:04:01,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-11 20:04:01,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-11 20:04:01,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-11 20:04:01,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-11 20:04:01,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-11 20:04:01,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-11 20:04:01,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-11 20:04:01,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-11 20:04:01,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-11 20:04:01,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-11 20:04:01,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-11 20:04:01,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-11 20:04:01,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-11 20:04:01,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-11 20:04:01,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-11 20:04:01,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-11 20:04:01,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-11 20:04:01,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-11 20:04:01,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-11 20:04:01,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-11 20:04:01,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-11 20:04:01,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-11 20:04:01,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-11 20:04:01,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-11 20:04:01,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-11 20:04:01,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-11 20:04:01,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-11 20:04:01,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-11 20:04:01,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-11 20:04:01,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-11 20:04:01,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-11 20:04:01,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-11 20:04:01,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-11 20:04:01,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-11 20:04:01,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-11 20:04:01,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-11 20:04:01,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-11 20:04:01,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-11 20:04:01,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-11 20:04:01,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-11 20:04:01,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-11 20:04:01,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-11 20:04:01,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-11 20:04:01,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-11 20:04:01,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-11 20:04:01,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-11 20:04:01,980 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.23782800878924E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0022222222222222222); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.23782800878924E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-11 20:04:01,981 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-11 20:04:01,981 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=3.503276924639066E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.23782800878924E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0022222222222222222); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.23782800878924E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-11 20:04:02,197 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 369 ms to try 5000 different iterations.  Found a solution that moves 42 regions; Going from a computed imbalance of 3.503276924639066E-5 to a new imbalance of 7.681847786030273E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0016412198797244363); MoveCostFunction : (multiplier=7.0, imbalance=0.004666666666666667); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-11 20:04:02,197 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1074760460:83 , srv1290362794:83 , srv1553447339:83 , srv1654328138:83 , srv1705544566:83 , srv1967615879:83 , srv2021797803:83 , srv507931530:83 , srv741481281:83 , srv863982248:83 , srv940466384:83 , srv1002788996:84 , srv1006649448:84 , srv1019592851:84 , srv1040958416:84 , srv1079019892:84 , srv1080597681:84 , srv1123530858:84 , srv1146770752:84 , srv1162362497:84 , srv124007561:84 , srv1292670493:84 , srv1299365920:84 , srv1304563804:84 , srv1359802974:84 , srv1362480562:84 , srv1380686652:84 , srv1405768853:84 , srv1417214683:84 , srv1417295236:84 , srv1418166188:84 , srv1425518296:84 , srv1496026944:84 , srv1527347704:84 , srv1574189060:84 , srv1579095824:84 , srv1680153861:84 , srv1680959874:84 , srv168414045:84 , srv1705160139:84 , srv1755996572:84 , srv1757387353:84 , srv1757464517:84 , srv1791939550:84 , srv1864184030:84 , srv1898126986:84 , srv1917333517:84 , srv1920549851:84 , srv1935849729:84 , srv1944862683:84 , srv2035951106:84 , srv2037271645:84 , srv2044352453:84 , srv2072151200:84 , srv2089823753:84 , srv2113145838:84 , srv220751828:84 , srv226112327:84 , srv237329048:84 , srv257062274:84 , srv263004393:84 , srv32955242:84 , srv401974939:84 , srv415488731:84 , srv439677912:84 , srv461085933:84 , srv47982392:84 , srv482493942:84 , srv505518009:84 , srv518931405:84 , srv567550137:84 , srv620021480:84 , srv621715005:84 , srv661363550:84 , srv687668001:84 , srv726554996:84 , srv731385793:84 , srv790266314:84 , srv826706471:84 , srv847884065:84 , srv865034866:84 , srv887414408:84 , srv938234584:84 , srv941335166:84 , srv959288445:84 , srv977213825:84 , srv996955990:84 , srv999859533:84 , srv105407453:85 , srv1080715976:85 , srv1402245096:85 , srv1405459484:85 , srv1686088543:85 , srv1924756842:85 , srv457722908:85 , srv503513922:85 , srv62415014:85 , srv740354971:85 , srv890341323:85 , srv1627274012:684 }
2023-04-11 20:04:02,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-11 20:04:02,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv507931530=74, srv457722908=68, srv1290362794=13, srv124007561=12, srv226112327=60, srv1405459484=21, srv1920549851=47, srv439677912=67, srv1579095824=31, srv865034866=90, srv1755996572=40, srv32955242=64, srv237329048=61, srv1079019892=6, srv263004393=63, srv401974939=65, srv1304563804=16, srv740354971=84, srv1680959874=35, srv1292670493=14, srv790266314=86, srv1917333517=46, srv2021797803=52, srv687668001=81, srv731385793=83, srv826706471=87, srv1006649448=1, srv47982392=70, srv62415014=79, srv726554996=82, srv2037271645=54, srv503513922=72, srv2113145838=58, srv1359802974=17, srv1002788996=0, srv1080597681=7, srv1362480562=18, srv1864184030=44, srv661363550=80, srv1757464517=42, srv1299365920=15, srv415488731=66, srv621715005=78, srv1040958416=3, srv890341323=92, srv1627274012=32, srv977213825=97, srv257062274=62, srv999859533=99, srv2072151200=56, srv482493942=71, srv959288445=96, srv1935849729=49, srv1123530858=9, srv1418166188=25, srv1654328138=33, srv1380686652=19, srv1425518296=26, srv461085933=69, srv168414045=36, srv2044352453=55, srv1527347704=28, srv1705544566=39, srv1757387353=41, srv1680153861=34, srv1417214683=23, srv1405768853=22, srv220751828=59, srv1574189060=30, srv1898126986=45, srv1791939550=43, srv2035951106=53, srv1074760460=5, srv863982248=89, srv1019592851=2, srv2089823753=57, srv505518009=73, srv518931405=75, srv1162362497=11, srv1146770752=10, srv1944862683=50, srv741481281=85, srv887414408=91, srv1686088543=37, srv1402245096=20, srv1705160139=38, srv996955990=98, srv105407453=4, srv941335166=95, srv938234584=93, srv940466384=94, srv620021480=77, srv1496026944=27, srv1553447339=29, srv1417295236=24, srv1080715976=8, srv1967615879=51, srv1924756842=48, srv567550137=76, srv847884065=88} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-11 20:04:02,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-11 20:04:02,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-11 20:04:02,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-11 20:04:02,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-11 20:04:02,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-11 20:04:02,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-11 20:04:02,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-11 20:04:02,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-11 20:04:02,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-11 20:04:02,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-11 20:04:02,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-11 20:04:02,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-11 20:04:02,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-11 20:04:02,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-11 20:04:02,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-11 20:04:02,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-11 20:04:02,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-11 20:04:02,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-11 20:04:02,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-11 20:04:02,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-11 20:04:02,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-11 20:04:02,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-11 20:04:02,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-11 20:04:02,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-11 20:04:02,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-11 20:04:02,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-11 20:04:02,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-11 20:04:02,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-11 20:04:02,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-11 20:04:02,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-11 20:04:02,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-11 20:04:02,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-11 20:04:02,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-11 20:04:02,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-11 20:04:02,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-11 20:04:02,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-11 20:04:02,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-11 20:04:02,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-11 20:04:02,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-11 20:04:02,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-11 20:04:02,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-11 20:04:02,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-11 20:04:02,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-11 20:04:02,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-11 20:04:02,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-11 20:04:02,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-11 20:04:02,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-11 20:04:02,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-11 20:04:02,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-11 20:04:02,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-11 20:04:02,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-11 20:04:02,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-11 20:04:02,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-11 20:04:02,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-11 20:04:02,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-11 20:04:02,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-11 20:04:02,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-11 20:04:02,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-11 20:04:02,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-11 20:04:02,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-11 20:04:02,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-11 20:04:02,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-11 20:04:02,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-11 20:04:02,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-11 20:04:02,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-11 20:04:02,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-11 20:04:02,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-11 20:04:02,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-11 20:04:02,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-11 20:04:02,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-11 20:04:02,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-11 20:04:02,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-11 20:04:02,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-11 20:04:02,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-11 20:04:02,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-11 20:04:02,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-11 20:04:02,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-11 20:04:02,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-11 20:04:02,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-11 20:04:02,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-11 20:04:02,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-11 20:04:02,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-11 20:04:02,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-11 20:04:02,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-11 20:04:02,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-11 20:04:02,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-11 20:04:02,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-11 20:04:02,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-11 20:04:02,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-11 20:04:02,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-11 20:04:02,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-11 20:04:02,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-11 20:04:02,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-11 20:04:02,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-11 20:04:02,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-11 20:04:02,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-11 20:04:02,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-11 20:04:02,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-11 20:04:02,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-11 20:04:02,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-11 20:04:02,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-11 20:04:02,386 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=7.387757500312559E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0016412198797244363); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-11 20:04:02,387 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1074760460:83 , srv1290362794:83 , srv1553447339:83 , srv1654328138:83 , srv1705544566:83 , srv1967615879:83 , srv2021797803:83 , srv507931530:83 , srv741481281:83 , srv863982248:83 , srv940466384:83 , srv1002788996:84 , srv1006649448:84 , srv1019592851:84 , srv1040958416:84 , srv1079019892:84 , srv1080597681:84 , srv1123530858:84 , srv1146770752:84 , srv1162362497:84 , srv124007561:84 , srv1292670493:84 , srv1299365920:84 , srv1304563804:84 , srv1359802974:84 , srv1362480562:84 , srv1380686652:84 , srv1405768853:84 , srv1417214683:84 , srv1417295236:84 , srv1418166188:84 , srv1425518296:84 , srv1496026944:84 , srv1527347704:84 , srv1574189060:84 , srv1579095824:84 , srv1680153861:84 , srv1680959874:84 , srv168414045:84 , srv1705160139:84 , srv1755996572:84 , srv1757387353:84 , srv1757464517:84 , srv1791939550:84 , srv1864184030:84 , srv1898126986:84 , srv1917333517:84 , srv1920549851:84 , srv1935849729:84 , srv1944862683:84 , srv2035951106:84 , srv2037271645:84 , srv2044352453:84 , srv2072151200:84 , srv2089823753:84 , srv2113145838:84 , srv220751828:84 , srv226112327:84 , srv237329048:84 , srv257062274:84 , srv263004393:84 , srv32955242:84 , srv401974939:84 , srv415488731:84 , srv439677912:84 , srv461085933:84 , srv47982392:84 , srv482493942:84 , srv505518009:84 , srv518931405:84 , srv567550137:84 , srv620021480:84 , srv621715005:84 , srv661363550:84 , srv687668001:84 , srv726554996:84 , srv731385793:84 , srv790266314:84 , srv826706471:84 , srv847884065:84 , srv865034866:84 , srv887414408:84 , srv938234584:84 , srv941335166:84 , srv959288445:84 , srv977213825:84 , srv996955990:84 , srv999859533:84 , srv105407453:85 , srv1080715976:85 , srv1402245096:85 , srv1405459484:85 , srv1686088543:85 , srv1924756842:85 , srv457722908:85 , srv503513922:85 , srv62415014:85 , srv740354971:85 , srv890341323:85 , srv1627274012:684 }
2023-04-11 20:04:02,485 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=889 (was 861) - SystemLoadAverage LEAK? -, ProcessCount=166 (was 166), AvailableMemoryMB=5126 (was 5720)
2023-04-11 20:04:02,513 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=889, ProcessCount=166, AvailableMemoryMB=5125
2023-04-11 20:04:02,514 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-11 20:04:02,514 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-04-11 20:04:02,529 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1022290285:84 , srv1030500662:84 , srv1105271282:84 , srv1312026656:84 , srv1421487961:84 , srv1429727204:84 , srv1491959163:84 , srv1535895478:84 , srv1622351674:84 , srv1633911787:84 , srv1710394272:84 , srv1764815768:84 , srv1861412347:84 , srv1938565654:84 , srv1946355778:84 , srv2022731126:84 , srv220478503:84 , srv229398268:84 , srv287354049:84 , srv295275617:84 , srv512066903:84 , srv581000314:84 , srv639439792:84 , srv675130148:84 , srv774877365:84 , srv787637052:84 , srv862991745:84 , srv883531330:84 , srv920973705:84 , srv1220124937:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-04-11 20:04:02,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-11 20:04:02,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1946355778=15, srv220478503=17, srv639439792=23, srv862991745=27, srv1220124937=3, srv1938565654=14, srv1710394272=11, srv774877365=25, srv1429727204=6, srv1535895478=8, srv883531330=28, srv675130148=24, srv1622351674=9, srv229398268=18, srv581000314=22, srv1633911787=10, srv1022290285=0, srv295275617=20, srv1105271282=2, srv287354049=19, srv920973705=29, srv1861412347=13, srv1030500662=1, srv787637052=26, srv1421487961=5, srv2022731126=16, srv1491959163=7, srv1764815768=12, srv1312026656=4, srv512066903=21} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-11 20:04:02,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-11 20:04:02,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-11 20:04:02,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-11 20:04:02,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-11 20:04:02,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-11 20:04:02,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-11 20:04:02,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-11 20:04:02,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-11 20:04:02,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-11 20:04:02,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-11 20:04:02,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-11 20:04:02,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-11 20:04:02,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-11 20:04:02,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-11 20:04:02,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-11 20:04:02,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-11 20:04:02,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-11 20:04:02,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-11 20:04:02,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-11 20:04:02,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-11 20:04:02,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-11 20:04:02,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-11 20:04:02,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-11 20:04:02,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-11 20:04:02,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-11 20:04:02,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-11 20:04:02,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-11 20:04:02,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-11 20:04:02,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-11 20:04:02,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-11 20:04:02,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-04-11 20:04:02,580 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1499193859856496, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-11 20:04:02,580 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912665434354202, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1499193859856496, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-04-11 20:06:02,563 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5276024 different iterations.  Found a solution that moves 1866 regions; Going from a computed imbalance of 0.9912665434354202 to a new imbalance of 4.3553370884861656E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6911111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-11 20:06:02,586 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1022290285:90 , srv1030500662:90 , srv1105271282:90 , srv1220124937:90 , srv1312026656:90 , srv1421487961:90 , srv1429727204:90 , srv1491959163:90 , srv1535895478:90 , srv1622351674:90 , srv1633911787:90 , srv1710394272:90 , srv1764815768:90 , srv1861412347:90 , srv1938565654:90 , srv1946355778:90 , srv2022731126:90 , srv220478503:90 , srv229398268:90 , srv287354049:90 , srv295275617:90 , srv512066903:90 , srv581000314:90 , srv639439792:90 , srv675130148:90 , srv774877365:90 , srv787637052:90 , srv862991745:90 , srv883531330:90 , srv920973705:90 }
2023-04-11 20:06:02,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-11 20:06:02,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1946355778=15, srv220478503=17, srv639439792=23, srv862991745=27, srv1220124937=3, srv1938565654=14, srv1710394272=11, srv774877365=25, srv1429727204=6, srv1535895478=8, srv883531330=28, srv675130148=24, srv1622351674=9, srv229398268=18, srv581000314=22, srv1633911787=10, srv1022290285=0, srv295275617=20, srv1105271282=2, srv287354049=19, srv920973705=29, srv1861412347=13, srv1030500662=1, srv787637052=26, srv1421487961=5, srv2022731126=16, srv1491959163=7, srv1764815768=12, srv1312026656=4, srv512066903=21} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-11 20:06:02,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-11 20:06:02,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-11 20:06:02,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-11 20:06:02,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-11 20:06:02,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-11 20:06:02,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-11 20:06:02,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-11 20:06:02,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-11 20:06:02,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-11 20:06:02,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-11 20:06:02,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-11 20:06:02,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-11 20:06:02,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-11 20:06:02,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-11 20:06:02,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-11 20:06:02,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-11 20:06:02,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-11 20:06:02,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-11 20:06:02,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-11 20:06:02,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-11 20:06:02,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-11 20:06:02,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-11 20:06:02,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-11 20:06:02,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-11 20:06:02,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-11 20:06:02,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-11 20:06:02,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-11 20:06:02,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-11 20:06:02,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-11 20:06:02,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-11 20:06:02,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-04-11 20:06:02,642 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-11 20:06:02,642 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1022290285:90 , srv1030500662:90 , srv1105271282:90 , srv1220124937:90 , srv1312026656:90 , srv1421487961:90 , srv1429727204:90 , srv1491959163:90 , srv1535895478:90 , srv1622351674:90 , srv1633911787:90 , srv1710394272:90 , srv1764815768:90 , srv1861412347:90 , srv1938565654:90 , srv1946355778:90 , srv2022731126:90 , srv220478503:90 , srv229398268:90 , srv287354049:90 , srv295275617:90 , srv512066903:90 , srv581000314:90 , srv639439792:90 , srv675130148:90 , srv774877365:90 , srv787637052:90 , srv862991745:90 , srv883531330:90 , srv920973705:90 }
2023-04-11 20:06:02,697 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=1042 (was 889) - SystemLoadAverage LEAK? -, ProcessCount=166 (was 166), AvailableMemoryMB=5370 (was 5125) - AvailableMemoryMB LEAK? -