2023-05-20 11:53:23,510 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-05-20 11:53:23,832 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-05-20 11:53:23,939 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-05-20 11:53:24,004 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-20 11:53:24,012 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-05-20 11:53:24,025 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=264, MaxFileDescriptor=60000, SystemLoadAverage=314, ProcessCount=172, AvailableMemoryMB=8437
2023-05-20 11:53:24,027 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-20 11:53:24,028 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-05-20 11:53:24,182 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1034569929:84 , srv1053520209:84 , srv1092120372:84 , srv1102158190:84 , srv1146069718:84 , srv1150112049:84 , srv1153495165:84 , srv115797057:84 , srv1197393494:84 , srv1238685582:84 , srv1242175377:84 , srv126238037:84 , srv1275876616:84 , srv1310487952:84 , srv1317918908:84 , srv1331933972:84 , srv133700292:84 , srv1360133608:84 , srv1382527809:84 , srv1398483244:84 , srv1414359853:84 , srv1436014362:84 , srv1502649928:84 , srv1561692122:84 , srv1571460287:84 , srv1616499762:84 , srv1643858781:84 , srv1680023164:84 , srv171310115:84 , srv1740982693:84 , srv1746415158:84 , srv1748009303:84 , srv1785525317:84 , srv179725825:84 , srv1803302329:84 , srv1814596231:84 , srv1834561407:84 , srv1840065156:84 , srv1908533427:84 , srv1923551984:84 , srv1930169276:84 , srv1942049455:84 , srv1963721096:84 , srv1982422810:84 , srv2011540237:84 , srv2028154353:84 , srv2029044175:84 , srv205997817:84 , srv2082636527:84 , srv2088302784:84 , srv2122440146:84 , srv220723889:84 , srv224016941:84 , srv234276450:84 , srv250126541:84 , srv268453464:84 , srv281378654:84 , srv293086790:84 , srv30137300:84 , srv348326691:84 , srv358284813:84 , srv359585169:84 , srv436454668:84 , srv442035058:84 , srv446262660:84 , srv516294141:84 , srv519865522:84 , srv574591354:84 , srv577059533:84 , srv625597293:84 , srv627554219:84 , srv63390132:84 , srv646248148:84 , srv662182466:84 , srv684841775:84 , srv706200279:84 , srv729083379:84 , srv734702838:84 , srv801165015:84 , srv808355478:84 , srv813545377:84 , srv820847162:84 , srv820944482:84 , srv822065411:84 , srv85684643:84 , srv866611632:84 , srv876336231:84 , srv886718456:84 , srv901018446:84 , srv902166551:84 , srv922944296:84 , srv936040515:84 , srv946755090:84 , srv950603311:84 , srv966472764:84 , srv970134403:84 , srv970722394:84 , srv985308168:84 , srv999602775:84 , srv1140678811:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-05-20 11:53:24,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-20 11:53:24,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv822065411=84, srv1238685582=10, srv1834561407=37, srv1803302329=35, srv205997817=48, srv1814596231=36, srv1382527809=19, srv179725825=34, srv2122440146=51, srv1436014362=22, srv966472764=95, srv946755090=93, srv1053520209=1, srv1034569929=0, srv866611632=86, srv30137300=59, srv1942049455=42, srv1360133608=18, srv820944482=83, srv126238037=12, srv646248148=73, srv115797057=8, srv1153495165=7, srv662182466=74, srv734702838=78, srv970134403=96, srv2029044175=47, srv813545377=81, srv936040515=92, srv1331933972=16, srv1242175377=11, srv2028154353=46, srv1930169276=41, srv706200279=76, srv1982422810=44, srv970722394=97, srv268453464=56, srv348326691=60, srv85684643=85, srv1317918908=15, srv1561692122=24, srv1840065156=38, srv574591354=68, srv1923551984=40, srv808355478=80, srv1643858781=27, srv1571460287=25, srv1746415158=31, srv171310115=29, srv876336231=87, srv801165015=79, srv446262660=65, srv902166551=90, srv2011540237=45, srv133700292=17, srv519865522=67, srv1140678811=4, srv281378654=57, srv224016941=53, srv250126541=55, srv1963721096=43, srv577059533=69, srv1102158190=3, srv1275876616=13, srv436454668=63, srv1748009303=32, srv1398483244=20, srv1785525317=33, srv293086790=58, srv2088302784=50, srv1616499762=26, srv2082636527=49, srv1740982693=30, srv1150112049=6, srv234276450=54, srv1092120372=2, srv625597293=70, srv627554219=71, srv950603311=94, srv820847162=82, srv901018446=89, srv985308168=98, srv999602775=99, srv1414359853=21, srv63390132=72, srv359585169=62, srv516294141=66, srv886718456=88, srv1502649928=23, srv684841775=75, srv220723889=52, srv1908533427=39, srv1197393494=9, srv442035058=64, srv1146069718=5, srv358284813=61, srv729083379=77, srv1310487952=14, srv1680023164=28, srv922944296=91} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-20 11:53:24,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-20 11:53:24,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-20 11:53:24,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-20 11:53:24,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-20 11:53:24,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-20 11:53:24,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-20 11:53:24,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-20 11:53:24,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-20 11:53:24,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-20 11:53:24,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-20 11:53:24,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-20 11:53:24,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-20 11:53:24,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-20 11:53:24,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-20 11:53:24,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-20 11:53:24,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-20 11:53:24,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-20 11:53:24,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-20 11:53:24,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-20 11:53:24,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-20 11:53:24,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-20 11:53:24,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-20 11:53:24,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-20 11:53:24,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-20 11:53:24,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-20 11:53:24,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-20 11:53:24,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-20 11:53:24,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-20 11:53:24,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-20 11:53:24,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-20 11:53:24,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-20 11:53:24,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-20 11:53:24,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-20 11:53:24,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-20 11:53:24,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-20 11:53:24,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-20 11:53:24,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-20 11:53:24,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-20 11:53:24,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-20 11:53:24,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-20 11:53:24,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-20 11:53:24,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-20 11:53:24,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-20 11:53:24,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-20 11:53:24,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-20 11:53:24,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-20 11:53:24,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-20 11:53:24,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-20 11:53:24,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-20 11:53:24,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-20 11:53:24,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-20 11:53:24,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-20 11:53:24,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-20 11:53:24,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-20 11:53:24,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-20 11:53:24,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-20 11:53:24,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-20 11:53:24,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-20 11:53:24,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-20 11:53:24,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-20 11:53:24,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-20 11:53:24,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-20 11:53:24,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-20 11:53:24,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-20 11:53:24,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-20 11:53:24,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-20 11:53:24,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-20 11:53:24,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-20 11:53:24,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-20 11:53:24,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-20 11:53:24,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-20 11:53:24,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-20 11:53:24,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-20 11:53:24,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-20 11:53:24,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-20 11:53:24,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-20 11:53:24,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-20 11:53:24,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-20 11:53:24,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-20 11:53:24,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-20 11:53:24,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-20 11:53:24,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-20 11:53:24,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-20 11:53:24,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-20 11:53:24,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-20 11:53:24,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-20 11:53:24,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-20 11:53:24,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-20 11:53:24,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-20 11:53:24,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-20 11:53:24,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-20 11:53:24,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-20 11:53:24,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-20 11:53:24,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-20 11:53:24,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-20 11:53:24,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-20 11:53:24,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-20 11:53:24,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-20 11:53:24,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-20 11:53:24,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-20 11:53:24,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-20 11:53:24,683 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-20 11:53:24,683 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-20 11:53:24,684 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-20 11:53:25,416 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 1083 ms to try 5000 different iterations.  Found a solution that moves 4711 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17765851902113655. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03959326819227486, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06119094876694492, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5234444444444445, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03959326819227486, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17216666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24616666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-20 11:53:25,439 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1275876616:82 , srv1310487952:82 , srv1414359853:82 , srv625597293:82 , srv999602775:82 , srv359585169:83 , srv574591354:83 , srv734702838:83 , srv1092120372:84 , srv1242175377:84 , srv1317918908:84 , srv1502649928:84 , srv1616499762:84 , srv171310115:84 , srv220723889:84 , srv224016941:84 , srv30137300:84 , srv446262660:84 , srv801165015:84 , srv813545377:84 , srv820847162:84 , srv970722394:84 , srv1034569929:85 , srv1053520209:85 , srv1153495165:85 , srv1360133608:85 , srv1561692122:85 , srv2088302784:85 , srv2122440146:85 , srv268453464:85 , srv519865522:85 , srv627554219:85 , srv662182466:85 , srv85684643:85 , srv886718456:85 , srv922944296:85 , srv1238685582:86 , srv1331933972:86 , srv1398483244:86 , srv1643858781:86 , srv1680023164:86 , srv1740982693:86 , srv1746415158:86 , srv1748009303:86 , srv179725825:86 , srv1942049455:86 , srv2028154353:86 , srv2029044175:86 , srv2082636527:86 , srv250126541:86 , srv820944482:86 , srv822065411:86 , srv866611632:86 , srv985308168:86 , srv1197393494:87 , srv1382527809:87 , srv1814596231:87 , srv1840065156:87 , srv1963721096:87 , srv2011540237:87 , srv348326691:87 , srv436454668:87 , srv63390132:87 , srv646248148:87 , srv706200279:87 , srv966472764:87 , srv1146069718:88 , srv1436014362:88 , srv1803302329:88 , srv1834561407:88 , srv1908533427:88 , srv1923551984:88 , srv205997817:88 , srv281378654:88 , srv358284813:88 , srv442035058:88 , srv516294141:88 , srv901018446:88 , srv902166551:88 , srv950603311:88 , srv1102158190:89 , srv1150112049:89 , srv126238037:89 , srv1930169276:89 , srv577059533:89 , srv729083379:89 , srv808355478:89 , srv876336231:89 , srv936040515:89 , srv946755090:89 , srv115797057:90 , srv133700292:90 , srv1785525317:90 , srv234276450:90 , srv1571460287:91 , srv684841775:91 , srv970134403:91 , srv1982422810:93 , srv293086790:93 , srv1140678811:442 }
2023-05-20 11:53:25,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-20 11:53:25,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv822065411=84, srv1238685582=10, srv1834561407=37, srv1803302329=35, srv205997817=48, srv1814596231=36, srv1382527809=19, srv179725825=34, srv2122440146=51, srv1436014362=22, srv966472764=95, srv946755090=93, srv1053520209=1, srv1034569929=0, srv866611632=86, srv30137300=59, srv1942049455=42, srv1360133608=18, srv820944482=83, srv126238037=12, srv646248148=73, srv115797057=8, srv1153495165=7, srv662182466=74, srv734702838=78, srv970134403=96, srv2029044175=47, srv813545377=81, srv936040515=92, srv1331933972=16, srv1242175377=11, srv2028154353=46, srv1930169276=41, srv706200279=76, srv1982422810=44, srv970722394=97, srv268453464=56, srv348326691=60, srv85684643=85, srv1317918908=15, srv1561692122=24, srv1840065156=38, srv574591354=68, srv1923551984=40, srv808355478=80, srv1643858781=27, srv1571460287=25, srv1746415158=31, srv171310115=29, srv876336231=87, srv801165015=79, srv446262660=65, srv902166551=90, srv2011540237=45, srv133700292=17, srv519865522=67, srv1140678811=4, srv281378654=57, srv224016941=53, srv250126541=55, srv1963721096=43, srv577059533=69, srv1102158190=3, srv1275876616=13, srv436454668=63, srv1748009303=32, srv1398483244=20, srv1785525317=33, srv293086790=58, srv2088302784=50, srv1616499762=26, srv2082636527=49, srv1740982693=30, srv1150112049=6, srv234276450=54, srv1092120372=2, srv625597293=70, srv627554219=71, srv950603311=94, srv820847162=82, srv901018446=89, srv985308168=98, srv999602775=99, srv1414359853=21, srv63390132=72, srv359585169=62, srv516294141=66, srv886718456=88, srv1502649928=23, srv684841775=75, srv220723889=52, srv1908533427=39, srv1197393494=9, srv442035058=64, srv1146069718=5, srv358284813=61, srv729083379=77, srv1310487952=14, srv1680023164=28, srv922944296=91} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-20 11:53:25,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-20 11:53:25,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-20 11:53:25,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-20 11:53:25,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-20 11:53:25,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-20 11:53:25,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-20 11:53:25,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-20 11:53:25,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-20 11:53:25,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-20 11:53:25,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-20 11:53:25,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-20 11:53:25,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-20 11:53:25,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-20 11:53:25,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-20 11:53:25,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-20 11:53:25,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-20 11:53:25,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-20 11:53:25,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-20 11:53:25,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-20 11:53:25,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-20 11:53:25,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-20 11:53:25,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-20 11:53:25,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-20 11:53:25,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-20 11:53:25,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-20 11:53:25,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-20 11:53:25,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-20 11:53:25,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-20 11:53:25,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-20 11:53:25,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-20 11:53:25,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-20 11:53:25,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-20 11:53:25,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-20 11:53:25,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-20 11:53:25,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-20 11:53:25,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-20 11:53:25,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-20 11:53:25,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-20 11:53:25,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-20 11:53:25,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-20 11:53:25,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-20 11:53:25,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-20 11:53:25,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-20 11:53:25,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-20 11:53:25,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-20 11:53:25,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-20 11:53:25,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-20 11:53:25,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-20 11:53:25,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-20 11:53:25,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-20 11:53:25,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-20 11:53:25,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-20 11:53:25,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-20 11:53:25,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-20 11:53:25,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-20 11:53:25,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-20 11:53:25,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-20 11:53:25,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-20 11:53:25,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-20 11:53:25,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-20 11:53:25,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-20 11:53:25,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-20 11:53:25,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-20 11:53:25,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-20 11:53:25,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-20 11:53:25,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-20 11:53:25,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-20 11:53:25,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-20 11:53:25,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-20 11:53:25,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-20 11:53:25,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-20 11:53:25,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-20 11:53:25,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-20 11:53:25,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-20 11:53:25,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-20 11:53:25,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-20 11:53:25,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-20 11:53:25,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-20 11:53:25,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-20 11:53:25,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-20 11:53:25,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-20 11:53:25,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-20 11:53:25,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-20 11:53:25,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-20 11:53:25,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-20 11:53:25,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-20 11:53:25,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-20 11:53:25,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-20 11:53:25,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-20 11:53:25,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-20 11:53:25,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-20 11:53:25,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-20 11:53:25,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-20 11:53:25,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-20 11:53:25,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-20 11:53:25,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-20 11:53:25,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-20 11:53:25,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-20 11:53:25,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-20 11:53:25,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-20 11:53:25,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-20 11:53:25,671 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03959326819227486, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06119094876694492, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03959326819227486, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17216666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24616666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-20 11:53:25,671 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-20 11:53:25,671 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17762553189408856, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03959326819227486, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06119094876694492, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03959326819227486, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17216666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24616666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-20 11:53:26,095 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 582 ms to try 5000 different iterations.  Found a solution that moves 2634 regions; Going from a computed imbalance of 0.17762553189408856 to a new imbalance of 0.014142546931728074. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023281702269296784); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04114794964297038, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2926666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023281702269296784); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010666666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04691666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-20 11:53:26,113 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1982422810:77 , srv684841775:80 , srv1571460287:81 , srv234276450:81 , srv936040515:81 , srv1102158190:82 , srv115797057:82 , srv1197393494:82 , srv133700292:82 , srv1785525317:82 , srv1908533427:82 , srv1923551984:82 , srv293086790:82 , srv946755090:82 , srv950603311:82 , srv970134403:82 , srv1146069718:83 , srv577059533:83 , srv63390132:83 , srv729083379:83 , srv902166551:83 , srv985308168:83 , srv1150112049:84 , srv1746415158:84 , srv1803302329:84 , srv1834561407:84 , srv1930169276:84 , srv250126541:84 , srv281378654:84 , srv436454668:84 , srv808355478:84 , srv876336231:84 , srv901018446:84 , srv1053520209:85 , srv1153495165:85 , srv126238037:85 , srv1382527809:85 , srv1643858781:85 , srv1680023164:85 , srv1748009303:85 , srv1963721096:85 , srv205997817:85 , srv348326691:85 , srv358284813:85 , srv516294141:85 , srv706200279:85 , srv966472764:85 , srv1034569929:86 , srv1238685582:86 , srv1331933972:86 , srv171310115:86 , srv1814596231:86 , srv1942049455:86 , srv2011540237:86 , srv2029044175:86 , srv2122440146:86 , srv442035058:86 , srv646248148:86 , srv662182466:86 , srv801165015:86 , srv813545377:86 , srv820944482:86 , srv822065411:86 , srv922944296:86 , srv970722394:86 , srv1242175377:87 , srv1317918908:87 , srv1360133608:87 , srv1502649928:87 , srv179725825:87 , srv2088302784:87 , srv220723889:87 , srv627554219:87 , srv820847162:87 , srv866611632:87 , srv1092120372:88 , srv1398483244:88 , srv1561692122:88 , srv1616499762:88 , srv1740982693:88 , srv1840065156:88 , srv2028154353:88 , srv2082636527:88 , srv268453464:88 , srv30137300:88 , srv519865522:88 , srv85684643:88 , srv886718456:88 , srv1414359853:89 , srv1436014362:89 , srv224016941:89 , srv359585169:89 , srv446262660:89 , srv574591354:89 , srv999602775:89 , srv1310487952:90 , srv734702838:90 , srv1275876616:91 , srv625597293:91 , srv1140678811:539 }
2023-05-20 11:53:26,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-20 11:53:26,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv822065411=84, srv1238685582=10, srv1834561407=37, srv1803302329=35, srv205997817=48, srv1814596231=36, srv1382527809=19, srv179725825=34, srv2122440146=51, srv1436014362=22, srv966472764=95, srv946755090=93, srv1053520209=1, srv1034569929=0, srv866611632=86, srv30137300=59, srv1942049455=42, srv1360133608=18, srv820944482=83, srv126238037=12, srv646248148=73, srv115797057=8, srv1153495165=7, srv662182466=74, srv734702838=78, srv970134403=96, srv2029044175=47, srv813545377=81, srv936040515=92, srv1331933972=16, srv1242175377=11, srv2028154353=46, srv1930169276=41, srv706200279=76, srv1982422810=44, srv970722394=97, srv268453464=56, srv348326691=60, srv85684643=85, srv1317918908=15, srv1561692122=24, srv1840065156=38, srv574591354=68, srv1923551984=40, srv808355478=80, srv1643858781=27, srv1571460287=25, srv1746415158=31, srv171310115=29, srv876336231=87, srv801165015=79, srv446262660=65, srv902166551=90, srv2011540237=45, srv133700292=17, srv519865522=67, srv1140678811=4, srv281378654=57, srv224016941=53, srv250126541=55, srv1963721096=43, srv577059533=69, srv1102158190=3, srv1275876616=13, srv436454668=63, srv1748009303=32, srv1398483244=20, srv1785525317=33, srv293086790=58, srv2088302784=50, srv1616499762=26, srv2082636527=49, srv1740982693=30, srv1150112049=6, srv234276450=54, srv1092120372=2, srv625597293=70, srv627554219=71, srv950603311=94, srv820847162=82, srv901018446=89, srv985308168=98, srv999602775=99, srv1414359853=21, srv63390132=72, srv359585169=62, srv516294141=66, srv886718456=88, srv1502649928=23, srv684841775=75, srv220723889=52, srv1908533427=39, srv1197393494=9, srv442035058=64, srv1146069718=5, srv358284813=61, srv729083379=77, srv1310487952=14, srv1680023164=28, srv922944296=91} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-20 11:53:26,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-20 11:53:26,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-20 11:53:26,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-20 11:53:26,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-20 11:53:26,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-20 11:53:26,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-20 11:53:26,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-20 11:53:26,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-20 11:53:26,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-20 11:53:26,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-20 11:53:26,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-20 11:53:26,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-20 11:53:26,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-20 11:53:26,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-20 11:53:26,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-20 11:53:26,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-20 11:53:26,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-20 11:53:26,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-20 11:53:26,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-20 11:53:26,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-20 11:53:26,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-20 11:53:26,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-20 11:53:26,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-20 11:53:26,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-20 11:53:26,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-20 11:53:26,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-20 11:53:26,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-20 11:53:26,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-20 11:53:26,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-20 11:53:26,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-20 11:53:26,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-20 11:53:26,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-20 11:53:26,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-20 11:53:26,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-20 11:53:26,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-20 11:53:26,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-20 11:53:26,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-20 11:53:26,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-20 11:53:26,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-20 11:53:26,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-20 11:53:26,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-20 11:53:26,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-20 11:53:26,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-20 11:53:26,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-20 11:53:26,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-20 11:53:26,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-20 11:53:26,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-20 11:53:26,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-20 11:53:26,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-20 11:53:26,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-20 11:53:26,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-20 11:53:26,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-20 11:53:26,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-20 11:53:26,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-20 11:53:26,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-20 11:53:26,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-20 11:53:26,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-20 11:53:26,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-20 11:53:26,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-20 11:53:26,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-20 11:53:26,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-20 11:53:26,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-20 11:53:26,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-20 11:53:26,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-20 11:53:26,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-20 11:53:26,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-20 11:53:26,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-20 11:53:26,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-20 11:53:26,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-20 11:53:26,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-20 11:53:26,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-20 11:53:26,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-20 11:53:26,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-20 11:53:26,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-20 11:53:26,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-20 11:53:26,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-20 11:53:26,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-20 11:53:26,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-20 11:53:26,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-20 11:53:26,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-20 11:53:26,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-20 11:53:26,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-20 11:53:26,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-20 11:53:26,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-20 11:53:26,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-20 11:53:26,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-20 11:53:26,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-20 11:53:26,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-20 11:53:26,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-20 11:53:26,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-20 11:53:26,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-20 11:53:26,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-20 11:53:26,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-20 11:53:26,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-20 11:53:26,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-20 11:53:26,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-20 11:53:26,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-20 11:53:26,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-20 11:53:26,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-20 11:53:26,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-20 11:53:26,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-20 11:53:26,278 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023281702269296784); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04114794964297038, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023281702269296784); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010666666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04691666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-20 11:53:26,278 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-20 11:53:26,278 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.014124103269523776, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023281702269296784); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04114794964297038, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023281702269296784); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010666666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04691666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-20 11:53:26,555 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 377 ms to try 5000 different iterations.  Found a solution that moves 1024 regions; Going from a computed imbalance of 0.014124103269523776 to a new imbalance of 9.291792650581821E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0060960430498352425); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009971902164189443); MoveCostFunction : (multiplier=7.0, imbalance=0.11377777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0060960430498352425); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.0E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-20 11:53:26,557 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1436014362:81 , srv126238037:83 , srv1840065156:83 , srv2082636527:83 , srv293086790:83 , srv1275876616:84 , srv1398483244:84 , srv1740982693:84 , srv1814596231:84 , srv1942049455:84 , srv2028154353:84 , srv205997817:84 , srv442035058:84 , srv519865522:84 , srv646248148:84 , srv734702838:84 , srv85684643:84 , srv866611632:84 , srv876336231:84 , srv970134403:84 , srv1150112049:85 , srv115797057:85 , srv1310487952:85 , srv1331933972:85 , srv133700292:85 , srv1360133608:85 , srv1382527809:85 , srv1561692122:85 , srv1616499762:85 , srv179725825:85 , srv1803302329:85 , srv1930169276:85 , srv1963721096:85 , srv2011540237:85 , srv268453464:85 , srv348326691:85 , srv358284813:85 , srv446262660:85 , srv516294141:85 , srv574591354:85 , srv577059533:85 , srv625597293:85 , srv627554219:85 , srv706200279:85 , srv808355478:85 , srv820944482:85 , srv822065411:85 , srv886718456:85 , srv901018446:85 , srv1034569929:86 , srv1092120372:86 , srv1102158190:86 , srv1146069718:86 , srv1238685582:86 , srv1242175377:86 , srv1317918908:86 , srv1414359853:86 , srv1502649928:86 , srv1571460287:86 , srv1680023164:86 , srv1748009303:86 , srv1785525317:86 , srv1834561407:86 , srv2088302784:86 , srv220723889:86 , srv224016941:86 , srv250126541:86 , srv281378654:86 , srv30137300:86 , srv359585169:86 , srv436454668:86 , srv684841775:86 , srv729083379:86 , srv820847162:86 , srv922944296:86 , srv946755090:86 , srv966472764:86 , srv1643858781:87 , srv171310115:87 , srv1746415158:87 , srv1923551984:87 , srv1982422810:87 , srv2029044175:87 , srv2122440146:87 , srv234276450:87 , srv63390132:87 , srv662182466:87 , srv813545377:87 , srv902166551:87 , srv936040515:87 , srv950603311:87 , srv970722394:87 , srv999602775:87 , srv1053520209:88 , srv1908533427:88 , srv801165015:88 , srv1153495165:89 , srv1197393494:89 , srv985308168:89 , srv1140678811:531 }
2023-05-20 11:53:26,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-20 11:53:26,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv822065411=84, srv1238685582=10, srv1834561407=37, srv1803302329=35, srv205997817=48, srv1814596231=36, srv1382527809=19, srv179725825=34, srv2122440146=51, srv1436014362=22, srv966472764=95, srv946755090=93, srv1053520209=1, srv1034569929=0, srv866611632=86, srv30137300=59, srv1942049455=42, srv1360133608=18, srv820944482=83, srv126238037=12, srv646248148=73, srv115797057=8, srv1153495165=7, srv662182466=74, srv734702838=78, srv970134403=96, srv2029044175=47, srv813545377=81, srv936040515=92, srv1331933972=16, srv1242175377=11, srv2028154353=46, srv1930169276=41, srv706200279=76, srv1982422810=44, srv970722394=97, srv268453464=56, srv348326691=60, srv85684643=85, srv1317918908=15, srv1561692122=24, srv1840065156=38, srv574591354=68, srv1923551984=40, srv808355478=80, srv1643858781=27, srv1571460287=25, srv1746415158=31, srv171310115=29, srv876336231=87, srv801165015=79, srv446262660=65, srv902166551=90, srv2011540237=45, srv133700292=17, srv519865522=67, srv1140678811=4, srv281378654=57, srv224016941=53, srv250126541=55, srv1963721096=43, srv577059533=69, srv1102158190=3, srv1275876616=13, srv436454668=63, srv1748009303=32, srv1398483244=20, srv1785525317=33, srv293086790=58, srv2088302784=50, srv1616499762=26, srv2082636527=49, srv1740982693=30, srv1150112049=6, srv234276450=54, srv1092120372=2, srv625597293=70, srv627554219=71, srv950603311=94, srv820847162=82, srv901018446=89, srv985308168=98, srv999602775=99, srv1414359853=21, srv63390132=72, srv359585169=62, srv516294141=66, srv886718456=88, srv1502649928=23, srv684841775=75, srv220723889=52, srv1908533427=39, srv1197393494=9, srv442035058=64, srv1146069718=5, srv358284813=61, srv729083379=77, srv1310487952=14, srv1680023164=28, srv922944296=91} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-20 11:53:26,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-20 11:53:26,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-20 11:53:26,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-20 11:53:26,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-20 11:53:26,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-20 11:53:26,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-20 11:53:26,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-20 11:53:26,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-20 11:53:26,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-20 11:53:26,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-20 11:53:26,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-20 11:53:26,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-20 11:53:26,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-20 11:53:26,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-20 11:53:26,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-20 11:53:26,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-20 11:53:26,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-20 11:53:26,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-20 11:53:26,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-20 11:53:26,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-20 11:53:26,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-20 11:53:26,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-20 11:53:26,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-20 11:53:26,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-20 11:53:26,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-20 11:53:26,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-20 11:53:26,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-20 11:53:26,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-20 11:53:26,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-20 11:53:26,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-20 11:53:26,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-20 11:53:26,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-20 11:53:26,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-20 11:53:26,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-20 11:53:26,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-20 11:53:26,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-20 11:53:26,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-20 11:53:26,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-20 11:53:26,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-20 11:53:26,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-20 11:53:26,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-20 11:53:26,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-20 11:53:26,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-20 11:53:26,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-20 11:53:26,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-20 11:53:26,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-20 11:53:26,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-20 11:53:26,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-20 11:53:26,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-20 11:53:26,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-20 11:53:26,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-20 11:53:26,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-20 11:53:26,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-20 11:53:26,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-20 11:53:26,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-20 11:53:26,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-20 11:53:26,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-20 11:53:26,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-20 11:53:26,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-20 11:53:26,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-20 11:53:26,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-20 11:53:26,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-20 11:53:26,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-20 11:53:26,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-20 11:53:26,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-20 11:53:26,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-20 11:53:26,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-20 11:53:26,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-20 11:53:26,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-20 11:53:26,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-20 11:53:26,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-20 11:53:26,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-20 11:53:26,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-20 11:53:26,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-20 11:53:26,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-20 11:53:26,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-20 11:53:26,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-20 11:53:26,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-20 11:53:26,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-20 11:53:26,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-20 11:53:26,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-20 11:53:26,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-20 11:53:26,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-20 11:53:26,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-20 11:53:26,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-20 11:53:26,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-20 11:53:26,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-20 11:53:26,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-20 11:53:26,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-20 11:53:26,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-20 11:53:26,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-20 11:53:26,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-20 11:53:26,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-20 11:53:26,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-20 11:53:26,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-20 11:53:26,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-20 11:53:26,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-20 11:53:26,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-20 11:53:26,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-20 11:53:26,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-20 11:53:26,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-20 11:53:26,695 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0060960430498352425); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009971902164189443); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0060960430498352425); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.0E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-20 11:53:26,696 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-20 11:53:26,696 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=9.220090638063978E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0060960430498352425); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009971902164189443); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0060960430498352425); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.0E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-20 11:53:26,947 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 346 ms to try 5000 different iterations.  Found a solution that moves 280 regions; Going from a computed imbalance of 9.220090638063978E-4 to a new imbalance of 2.1988764737006278E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.23782800878924E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0022222222222222222); MoveCostFunction : (multiplier=7.0, imbalance=0.03111111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.23782800878924E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-20 11:53:26,948 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1153495165:83 , srv1398483244:83 , srv2029044175:83 , srv224016941:83 , srv268453464:83 , srv706200279:83 , srv1053520209:84 , srv1092120372:84 , srv1197393494:84 , srv1238685582:84 , srv1436014362:84 , srv1561692122:84 , srv1571460287:84 , srv1643858781:84 , srv1740982693:84 , srv1748009303:84 , srv1785525317:84 , srv179725825:84 , srv1840065156:84 , srv1908533427:84 , srv1930169276:84 , srv1982422810:84 , srv2011540237:84 , srv2028154353:84 , srv2088302784:84 , srv234276450:84 , srv281378654:84 , srv293086790:84 , srv30137300:84 , srv359585169:84 , srv442035058:84 , srv446262660:84 , srv516294141:84 , srv574591354:84 , srv625597293:84 , srv63390132:84 , srv729083379:84 , srv801165015:84 , srv808355478:84 , srv886718456:84 , srv902166551:84 , srv936040515:84 , srv966472764:84 , srv985308168:84 , srv999602775:84 , srv1034569929:85 , srv1146069718:85 , srv1150112049:85 , srv115797057:85 , srv1242175377:85 , srv126238037:85 , srv1275876616:85 , srv1317918908:85 , srv1331933972:85 , srv133700292:85 , srv1360133608:85 , srv1382527809:85 , srv1414359853:85 , srv1616499762:85 , srv1680023164:85 , srv171310115:85 , srv1746415158:85 , srv1814596231:85 , srv1834561407:85 , srv1923551984:85 , srv1963721096:85 , srv205997817:85 , srv2082636527:85 , srv2122440146:85 , srv220723889:85 , srv348326691:85 , srv358284813:85 , srv519865522:85 , srv577059533:85 , srv627554219:85 , srv646248148:85 , srv662182466:85 , srv684841775:85 , srv734702838:85 , srv813545377:85 , srv820847162:85 , srv820944482:85 , srv822065411:85 , srv85684643:85 , srv866611632:85 , srv876336231:85 , srv901018446:85 , srv922944296:85 , srv946755090:85 , srv950603311:85 , srv970134403:85 , srv970722394:85 , srv1102158190:86 , srv1310487952:86 , srv1502649928:86 , srv1803302329:86 , srv1942049455:86 , srv250126541:86 , srv436454668:86 , srv1140678811:629 }
2023-05-20 11:53:26,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-20 11:53:26,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv822065411=84, srv1238685582=10, srv1834561407=37, srv1803302329=35, srv205997817=48, srv1814596231=36, srv1382527809=19, srv179725825=34, srv2122440146=51, srv1436014362=22, srv966472764=95, srv946755090=93, srv1053520209=1, srv1034569929=0, srv866611632=86, srv30137300=59, srv1942049455=42, srv1360133608=18, srv820944482=83, srv126238037=12, srv646248148=73, srv115797057=8, srv1153495165=7, srv662182466=74, srv734702838=78, srv970134403=96, srv2029044175=47, srv813545377=81, srv936040515=92, srv1331933972=16, srv1242175377=11, srv2028154353=46, srv1930169276=41, srv706200279=76, srv1982422810=44, srv970722394=97, srv268453464=56, srv348326691=60, srv85684643=85, srv1317918908=15, srv1561692122=24, srv1840065156=38, srv574591354=68, srv1923551984=40, srv808355478=80, srv1643858781=27, srv1571460287=25, srv1746415158=31, srv171310115=29, srv876336231=87, srv801165015=79, srv446262660=65, srv902166551=90, srv2011540237=45, srv133700292=17, srv519865522=67, srv1140678811=4, srv281378654=57, srv224016941=53, srv250126541=55, srv1963721096=43, srv577059533=69, srv1102158190=3, srv1275876616=13, srv436454668=63, srv1748009303=32, srv1398483244=20, srv1785525317=33, srv293086790=58, srv2088302784=50, srv1616499762=26, srv2082636527=49, srv1740982693=30, srv1150112049=6, srv234276450=54, srv1092120372=2, srv625597293=70, srv627554219=71, srv950603311=94, srv820847162=82, srv901018446=89, srv985308168=98, srv999602775=99, srv1414359853=21, srv63390132=72, srv359585169=62, srv516294141=66, srv886718456=88, srv1502649928=23, srv684841775=75, srv220723889=52, srv1908533427=39, srv1197393494=9, srv442035058=64, srv1146069718=5, srv358284813=61, srv729083379=77, srv1310487952=14, srv1680023164=28, srv922944296=91} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-20 11:53:26,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-20 11:53:26,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-20 11:53:26,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-20 11:53:26,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-20 11:53:26,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-20 11:53:26,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-20 11:53:26,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-20 11:53:26,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-20 11:53:26,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-20 11:53:26,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-20 11:53:26,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-20 11:53:26,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-20 11:53:26,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-20 11:53:26,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-20 11:53:26,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-20 11:53:26,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-20 11:53:26,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-20 11:53:26,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-20 11:53:26,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-20 11:53:26,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-20 11:53:26,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-20 11:53:26,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-20 11:53:26,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-20 11:53:26,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-20 11:53:26,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-20 11:53:26,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-20 11:53:26,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-20 11:53:26,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-20 11:53:26,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-20 11:53:26,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-20 11:53:26,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-20 11:53:26,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-20 11:53:26,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-20 11:53:26,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-20 11:53:26,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-20 11:53:26,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-20 11:53:26,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-20 11:53:26,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-20 11:53:26,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-20 11:53:26,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-20 11:53:26,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-20 11:53:26,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-20 11:53:26,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-20 11:53:26,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-20 11:53:26,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-20 11:53:26,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-20 11:53:26,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-20 11:53:26,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-20 11:53:26,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-20 11:53:26,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-20 11:53:26,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-20 11:53:26,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-20 11:53:26,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-20 11:53:26,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-20 11:53:26,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-20 11:53:26,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-20 11:53:26,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-20 11:53:26,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-20 11:53:26,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-20 11:53:26,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-20 11:53:26,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-20 11:53:26,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-20 11:53:26,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-20 11:53:26,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-20 11:53:26,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-20 11:53:26,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-20 11:53:26,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-20 11:53:26,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-20 11:53:26,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-20 11:53:26,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-20 11:53:26,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-20 11:53:26,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-20 11:53:26,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-20 11:53:26,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-20 11:53:26,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-20 11:53:26,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-20 11:53:26,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-20 11:53:26,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-20 11:53:26,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-20 11:53:26,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-20 11:53:26,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-20 11:53:26,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-20 11:53:26,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-20 11:53:26,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-20 11:53:26,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-20 11:53:26,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-20 11:53:26,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-20 11:53:26,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-20 11:53:26,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-20 11:53:26,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-20 11:53:26,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-20 11:53:26,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-20 11:53:26,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-20 11:53:26,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-20 11:53:26,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-20 11:53:26,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-20 11:53:26,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-20 11:53:26,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-20 11:53:26,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-20 11:53:26,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-20 11:53:26,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-20 11:53:27,072 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=2.002816283222151E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.23782800878924E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0022222222222222222); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.23782800878924E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-20 11:53:27,072 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1153495165:83 , srv1398483244:83 , srv2029044175:83 , srv224016941:83 , srv268453464:83 , srv706200279:83 , srv1053520209:84 , srv1092120372:84 , srv1197393494:84 , srv1238685582:84 , srv1436014362:84 , srv1561692122:84 , srv1571460287:84 , srv1643858781:84 , srv1740982693:84 , srv1748009303:84 , srv1785525317:84 , srv179725825:84 , srv1840065156:84 , srv1908533427:84 , srv1930169276:84 , srv1982422810:84 , srv2011540237:84 , srv2028154353:84 , srv2088302784:84 , srv234276450:84 , srv281378654:84 , srv293086790:84 , srv30137300:84 , srv359585169:84 , srv442035058:84 , srv446262660:84 , srv516294141:84 , srv574591354:84 , srv625597293:84 , srv63390132:84 , srv729083379:84 , srv801165015:84 , srv808355478:84 , srv886718456:84 , srv902166551:84 , srv936040515:84 , srv966472764:84 , srv985308168:84 , srv999602775:84 , srv1034569929:85 , srv1146069718:85 , srv1150112049:85 , srv115797057:85 , srv1242175377:85 , srv126238037:85 , srv1275876616:85 , srv1317918908:85 , srv1331933972:85 , srv133700292:85 , srv1360133608:85 , srv1382527809:85 , srv1414359853:85 , srv1616499762:85 , srv1680023164:85 , srv171310115:85 , srv1746415158:85 , srv1814596231:85 , srv1834561407:85 , srv1923551984:85 , srv1963721096:85 , srv205997817:85 , srv2082636527:85 , srv2122440146:85 , srv220723889:85 , srv348326691:85 , srv358284813:85 , srv519865522:85 , srv577059533:85 , srv627554219:85 , srv646248148:85 , srv662182466:85 , srv684841775:85 , srv734702838:85 , srv813545377:85 , srv820847162:85 , srv820944482:85 , srv822065411:85 , srv85684643:85 , srv866611632:85 , srv876336231:85 , srv901018446:85 , srv922944296:85 , srv946755090:85 , srv950603311:85 , srv970134403:85 , srv970722394:85 , srv1102158190:86 , srv1310487952:86 , srv1502649928:86 , srv1803302329:86 , srv1942049455:86 , srv250126541:86 , srv436454668:86 , srv1140678811:629 }
2023-05-20 11:53:27,102 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=264 (was 264), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=345 (was 314) - SystemLoadAverage LEAK? -, ProcessCount=172 (was 172), AvailableMemoryMB=7803 (was 8437)
2023-05-20 11:53:27,111 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=264, MaxFileDescriptor=60000, SystemLoadAverage=345, ProcessCount=172, AvailableMemoryMB=7802
2023-05-20 11:53:27,111 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-20 11:53:27,111 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-05-20 11:53:27,120 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1035865799:84 , srv1185539103:84 , srv124583616:84 , srv1257383758:84 , srv1272944208:84 , srv1393870801:84 , srv1610194196:84 , srv1870520073:84 , srv1909183130:84 , srv1941305927:84 , srv1949854726:84 , srv1969269843:84 , srv2021795962:84 , srv2092238366:84 , srv2145863670:84 , srv247991118:84 , srv346804273:84 , srv377605250:84 , srv416558699:84 , srv454203479:84 , srv466607970:84 , srv475745572:84 , srv481162977:84 , srv484947369:84 , srv60134571:84 , srv61941874:84 , srv692297855:84 , srv785540971:84 , srv86578851:84 , srv105295208:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-05-20 11:53:27,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-20 11:53:27,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1610194196=7, srv105295208=1, srv481162977=23, srv416558699=19, srv60134571=25, srv2021795962=13, srv247991118=16, srv346804273=17, srv1272944208=5, srv466607970=21, srv1870520073=8, srv1969269843=12, srv475745572=22, srv86578851=29, srv1035865799=0, srv61941874=26, srv692297855=27, srv785540971=28, srv1185539103=2, srv1941305927=10, srv1949854726=11, srv2145863670=15, srv454203479=20, srv2092238366=14, srv1257383758=4, srv124583616=3, srv377605250=18, srv484947369=24, srv1909183130=9, srv1393870801=6} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-20 11:53:27,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-20 11:53:27,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-20 11:53:27,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-20 11:53:27,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-20 11:53:27,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-20 11:53:27,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-20 11:53:27,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-20 11:53:27,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-20 11:53:27,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-20 11:53:27,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-20 11:53:27,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-20 11:53:27,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-20 11:53:27,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-20 11:53:27,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-20 11:53:27,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-20 11:53:27,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-20 11:53:27,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-20 11:53:27,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-20 11:53:27,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-20 11:53:27,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-20 11:53:27,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-20 11:53:27,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-20 11:53:27,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-20 11:53:27,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-20 11:53:27,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-20 11:53:27,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-20 11:53:27,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-20 11:53:27,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-20 11:53:27,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-20 11:53:27,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-20 11:53:27,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-05-20 11:53:27,147 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0465750945581167, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-20 11:53:27,147 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912339799866416, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0465750945581167, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-05-20 11:55:27,133 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6885953 different iterations.  Found a solution that moves 1848 regions; Going from a computed imbalance of 0.9912339799866416 to a new imbalance of 4.313324190526491E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6844444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-20 11:55:27,141 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1035865799:90 , srv105295208:90 , srv1185539103:90 , srv124583616:90 , srv1257383758:90 , srv1272944208:90 , srv1393870801:90 , srv1610194196:90 , srv1870520073:90 , srv1909183130:90 , srv1941305927:90 , srv1949854726:90 , srv1969269843:90 , srv2021795962:90 , srv2092238366:90 , srv2145863670:90 , srv247991118:90 , srv346804273:90 , srv377605250:90 , srv416558699:90 , srv454203479:90 , srv466607970:90 , srv475745572:90 , srv481162977:90 , srv484947369:90 , srv60134571:90 , srv61941874:90 , srv692297855:90 , srv785540971:90 , srv86578851:90 }
2023-05-20 11:55:27,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-20 11:55:27,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1610194196=7, srv105295208=1, srv481162977=23, srv416558699=19, srv60134571=25, srv2021795962=13, srv247991118=16, srv346804273=17, srv1272944208=5, srv466607970=21, srv1870520073=8, srv1969269843=12, srv475745572=22, srv86578851=29, srv1035865799=0, srv61941874=26, srv692297855=27, srv785540971=28, srv1185539103=2, srv1941305927=10, srv1949854726=11, srv2145863670=15, srv454203479=20, srv2092238366=14, srv1257383758=4, srv124583616=3, srv377605250=18, srv484947369=24, srv1909183130=9, srv1393870801=6} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-20 11:55:27,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-20 11:55:27,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-20 11:55:27,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-20 11:55:27,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-20 11:55:27,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-20 11:55:27,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-20 11:55:27,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-20 11:55:27,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-20 11:55:27,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-20 11:55:27,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-20 11:55:27,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-20 11:55:27,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-20 11:55:27,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-20 11:55:27,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-20 11:55:27,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-20 11:55:27,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-20 11:55:27,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-20 11:55:27,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-20 11:55:27,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-20 11:55:27,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-20 11:55:27,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-20 11:55:27,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-20 11:55:27,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-20 11:55:27,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-20 11:55:27,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-20 11:55:27,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-20 11:55:27,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-20 11:55:27,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-20 11:55:27,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-20 11:55:27,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-20 11:55:27,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-05-20 11:55:27,159 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-20 11:55:27,159 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1035865799:90 , srv105295208:90 , srv1185539103:90 , srv124583616:90 , srv1257383758:90 , srv1272944208:90 , srv1393870801:90 , srv1610194196:90 , srv1870520073:90 , srv1909183130:90 , srv1941305927:90 , srv1949854726:90 , srv1969269843:90 , srv2021795962:90 , srv2092238366:90 , srv2145863670:90 , srv247991118:90 , srv346804273:90 , srv377605250:90 , srv416558699:90 , srv454203479:90 , srv466607970:90 , srv475745572:90 , srv481162977:90 , srv484947369:90 , srv60134571:90 , srv61941874:90 , srv692297855:90 , srv785540971:90 , srv86578851:90 }
2023-05-20 11:55:27,171 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=264 (was 264), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=243 (was 345), ProcessCount=169 (was 172), AvailableMemoryMB=7610 (was 7802)