2023-04-27 23:55:38,492 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-04-27 23:55:38,876 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-04-27 23:55:39,024 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-04-27 23:55:39,112 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-27 23:55:39,122 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-04-27 23:55:39,138 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=437, ProcessCount=171, AvailableMemoryMB=9394
2023-04-27 23:55:39,141 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-27 23:55:39,141 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-04-27 23:55:39,310 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1052391350:84 , srv1061286597:84 , srv107361962:84 , srv1109957176:84 , srv111259787:84 , srv1134916488:84 , srv1146912502:84 , srv1160416620:84 , srv1225818519:84 , srv1239681650:84 , srv1247334324:84 , srv128646128:84 , srv1311185486:84 , srv1421815941:84 , srv143504573:84 , srv1500632723:84 , srv1508387942:84 , srv1515753254:84 , srv1521400614:84 , srv1548424272:84 , srv1563005908:84 , srv1575352750:84 , srv157956004:84 , srv1620935245:84 , srv1630677763:84 , srv1635032696:84 , srv1636231149:84 , srv1637925814:84 , srv1658859122:84 , srv1674505315:84 , srv1686802060:84 , srv1694326431:84 , srv1726768514:84 , srv1729353179:84 , srv1797829109:84 , srv1857037615:84 , srv1880024035:84 , srv188408159:84 , srv1885391765:84 , srv1891324134:84 , srv1892847137:84 , srv1921850322:84 , srv1954474731:84 , srv1963901146:84 , srv1969224337:84 , srv197053933:84 , srv2030171466:84 , srv2048253829:84 , srv2059829393:84 , srv2065772104:84 , srv2068925692:84 , srv2078478244:84 , srv2089623367:84 , srv2091865907:84 , srv2106568965:84 , srv2106667467:84 , srv2137824743:84 , srv2772161:84 , srv291264504:84 , srv304618238:84 , srv307548559:84 , srv323570204:84 , srv342024123:84 , srv351723097:84 , srv377425857:84 , srv40515242:84 , srv417083115:84 , srv433535584:84 , srv459878999:84 , srv464779117:84 , srv492505027:84 , srv495097909:84 , srv498651332:84 , srv508811828:84 , srv547045250:84 , srv55108175:84 , srv606120520:84 , srv614293029:84 , srv641545053:84 , srv654847029:84 , srv661620231:84 , srv682204729:84 , srv690312031:84 , srv747989542:84 , srv757529721:84 , srv759498598:84 , srv775604873:84 , srv798549144:84 , srv813267045:84 , srv861703114:84 , srv88104840:84 , srv884113348:84 , srv886991066:84 , srv936693421:84 , srv956690517:84 , srv96349630:84 , srv971362344:84 , srv987584409:84 , srv990101711:84 , srv1993853609:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-04-27 23:55:39,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-27 23:55:39,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1857037615=35, srv1954474731=42, srv2030171466=47, srv1575352750=21, srv1247334324=10, srv342024123=63, srv1637925814=27, srv1146912502=6, srv1921850322=41, srv886991066=93, srv1635032696=25, srv1694326431=31, srv2059829393=49, srv492505027=71, srv1620935245=23, srv304618238=60, srv1160416620=7, srv498651332=73, srv990101711=99, srv1797829109=34, srv188408159=37, srv2068925692=51, srv747989542=84, srv661620231=81, srv1421815941=13, srv323570204=62, srv1508387942=16, srv2091865907=54, srv641545053=79, srv307548559=61, srv1521400614=18, srv1636231149=26, srv936693421=94, srv1993853609=46, srv88104840=91, srv1061286597=1, srv157956004=22, srv606120520=77, srv861703114=90, srv55108175=76, srv377425857=65, srv614293029=78, srv495097909=72, srv1225818519=8, srv1134916488=5, srv1892847137=40, srv956690517=95, srv1548424272=19, srv1726768514=32, srv690312031=83, srv111259787=4, srv2089623367=53, srv654847029=80, srv1891324134=39, srv2137824743=57, srv1563005908=20, srv197053933=45, srv1963901146=43, srv987584409=98, srv1880024035=36, srv971362344=97, srv1515753254=17, srv757529721=85, srv759498598=86, srv798549144=88, srv1686802060=30, srv775604873=87, srv2078478244=52, srv1109957176=3, srv2106568965=55, srv1239681650=9, srv433535584=68, srv96349630=96, srv884113348=92, srv417083115=67, srv2772161=58, srv291264504=59, srv1052391350=0, srv40515242=66, srv351723097=64, srv2106667467=56, srv128646128=11, srv508811828=74, srv1500632723=15, srv2048253829=48, srv459878999=69, srv682204729=82, srv464779117=70, srv1969224337=44, srv547045250=75, srv1658859122=28, srv1674505315=29, srv2065772104=50, srv1729353179=33, srv1885391765=38, srv143504573=14, srv813267045=89, srv1311185486=12, srv107361962=2, srv1630677763=24} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-27 23:55:39,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-27 23:55:39,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-27 23:55:39,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-27 23:55:39,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-27 23:55:39,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-27 23:55:39,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-27 23:55:39,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-27 23:55:39,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-27 23:55:39,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-27 23:55:39,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-27 23:55:39,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-27 23:55:39,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-27 23:55:39,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-27 23:55:39,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-27 23:55:39,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-27 23:55:39,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-27 23:55:39,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-27 23:55:39,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-27 23:55:39,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-27 23:55:39,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-27 23:55:39,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-27 23:55:39,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-27 23:55:39,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-27 23:55:39,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-27 23:55:39,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-27 23:55:39,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-27 23:55:39,503 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-27 23:55:39,503 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-27 23:55:39,504 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-27 23:55:39,504 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-27 23:55:39,504 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-27 23:55:39,504 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-27 23:55:39,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-27 23:55:39,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-27 23:55:39,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-27 23:55:39,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-27 23:55:39,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-27 23:55:39,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-27 23:55:39,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-27 23:55:39,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-27 23:55:39,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-27 23:55:39,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-27 23:55:39,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-27 23:55:39,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-27 23:55:39,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-27 23:55:39,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-27 23:55:39,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-27 23:55:39,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-27 23:55:39,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-27 23:55:39,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-27 23:55:39,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-27 23:55:39,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-27 23:55:39,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-27 23:55:39,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-27 23:55:39,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-27 23:55:39,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-27 23:55:39,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-27 23:55:39,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-27 23:55:39,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-27 23:55:39,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-27 23:55:39,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-27 23:55:39,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-27 23:55:39,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-27 23:55:39,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-27 23:55:39,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-27 23:55:39,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-27 23:55:39,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-27 23:55:39,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-27 23:55:39,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-27 23:55:39,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-27 23:55:39,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-27 23:55:39,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-27 23:55:39,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-27 23:55:39,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-27 23:55:39,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-27 23:55:39,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-27 23:55:39,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-27 23:55:39,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-27 23:55:39,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-27 23:55:39,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-27 23:55:39,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-27 23:55:39,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-27 23:55:39,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-27 23:55:39,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-27 23:55:39,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-27 23:55:39,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-27 23:55:39,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-27 23:55:39,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-27 23:55:39,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-27 23:55:39,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-27 23:55:39,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-27 23:55:39,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-27 23:55:39,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-27 23:55:39,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-27 23:55:39,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-27 23:55:39,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-27 23:55:39,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-27 23:55:39,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-27 23:55:39,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-27 23:55:39,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-27 23:55:39,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-27 23:55:39,762 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-27 23:55:39,762 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-27 23:55:39,763 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-27 23:55:40,138 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 568 ms to try 5000 different iterations.  Found a solution that moves 4684 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17606813909260166. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039834737791963476, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05433198031820134, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5204444444444445, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039834737791963476, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17141666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23633333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-27 23:55:40,168 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1969224337:80 , srv1857037615:81 , srv88104840:81 , srv1160416620:83 , srv1637925814:83 , srv197053933:83 , srv2068925692:83 , srv2078478244:83 , srv2089623367:83 , srv377425857:83 , srv55108175:83 , srv661620231:83 , srv682204729:83 , srv775604873:83 , srv990101711:83 , srv1134916488:84 , srv1146912502:84 , srv1500632723:84 , srv1891324134:84 , srv2106667467:84 , srv614293029:84 , srv759498598:84 , srv971362344:84 , srv1109957176:85 , srv128646128:85 , srv143504573:85 , srv157956004:85 , srv1630677763:85 , srv1658859122:85 , srv291264504:85 , srv433535584:85 , srv884113348:85 , srv956690517:85 , srv96349630:85 , srv1052391350:86 , srv1225818519:86 , srv1311185486:86 , srv1515753254:86 , srv1548424272:86 , srv1674505315:86 , srv1694326431:86 , srv1797829109:86 , srv188408159:86 , srv1892847137:86 , srv1963901146:86 , srv304618238:86 , srv417083115:86 , srv492505027:86 , srv508811828:86 , srv606120520:86 , srv641545053:86 , srv798549144:86 , srv936693421:86 , srv1239681650:87 , srv1563005908:87 , srv1885391765:87 , srv2030171466:87 , srv2059829393:87 , srv2106568965:87 , srv307548559:87 , srv459878999:87 , srv654847029:87 , srv690312031:87 , srv757529721:87 , srv886991066:87 , srv107361962:88 , srv111259787:88 , srv1247334324:88 , srv1421815941:88 , srv1521400614:88 , srv1575352750:88 , srv1636231149:88 , srv1729353179:88 , srv1954474731:88 , srv2137824743:88 , srv342024123:88 , srv351723097:88 , srv40515242:88 , srv498651332:88 , srv547045250:88 , srv1508387942:89 , srv1620935245:89 , srv2048253829:89 , srv323570204:89 , srv464779117:89 , srv747989542:89 , srv813267045:89 , srv861703114:89 , srv987584409:89 , srv1635032696:90 , srv1686802060:90 , srv1726768514:90 , srv2065772104:90 , srv1921850322:91 , srv2772161:91 , srv495097909:91 , srv1061286597:92 , srv1880024035:92 , srv2091865907:95 , srv1993853609:444 }
2023-04-27 23:55:40,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-27 23:55:40,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1857037615=35, srv1954474731=42, srv2030171466=47, srv1575352750=21, srv1247334324=10, srv342024123=63, srv1637925814=27, srv1146912502=6, srv1921850322=41, srv886991066=93, srv1635032696=25, srv1694326431=31, srv2059829393=49, srv492505027=71, srv1620935245=23, srv304618238=60, srv1160416620=7, srv498651332=73, srv990101711=99, srv1797829109=34, srv188408159=37, srv2068925692=51, srv747989542=84, srv661620231=81, srv1421815941=13, srv323570204=62, srv1508387942=16, srv2091865907=54, srv641545053=79, srv307548559=61, srv1521400614=18, srv1636231149=26, srv936693421=94, srv1993853609=46, srv88104840=91, srv1061286597=1, srv157956004=22, srv606120520=77, srv861703114=90, srv55108175=76, srv377425857=65, srv614293029=78, srv495097909=72, srv1225818519=8, srv1134916488=5, srv1892847137=40, srv956690517=95, srv1548424272=19, srv1726768514=32, srv690312031=83, srv111259787=4, srv2089623367=53, srv654847029=80, srv1891324134=39, srv2137824743=57, srv1563005908=20, srv197053933=45, srv1963901146=43, srv987584409=98, srv1880024035=36, srv971362344=97, srv1515753254=17, srv757529721=85, srv759498598=86, srv798549144=88, srv1686802060=30, srv775604873=87, srv2078478244=52, srv1109957176=3, srv2106568965=55, srv1239681650=9, srv433535584=68, srv96349630=96, srv884113348=92, srv417083115=67, srv2772161=58, srv291264504=59, srv1052391350=0, srv40515242=66, srv351723097=64, srv2106667467=56, srv128646128=11, srv508811828=74, srv1500632723=15, srv2048253829=48, srv459878999=69, srv682204729=82, srv464779117=70, srv1969224337=44, srv547045250=75, srv1658859122=28, srv1674505315=29, srv2065772104=50, srv1729353179=33, srv1885391765=38, srv143504573=14, srv813267045=89, srv1311185486=12, srv107361962=2, srv1630677763=24} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-27 23:55:40,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-27 23:55:40,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-27 23:55:40,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-27 23:55:40,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-27 23:55:40,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-27 23:55:40,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-27 23:55:40,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-27 23:55:40,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-27 23:55:40,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-27 23:55:40,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-27 23:55:40,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-27 23:55:40,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-27 23:55:40,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-27 23:55:40,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-27 23:55:40,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-27 23:55:40,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-27 23:55:40,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-27 23:55:40,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-27 23:55:40,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-27 23:55:40,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-27 23:55:40,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-27 23:55:40,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-27 23:55:40,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-27 23:55:40,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-27 23:55:40,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-27 23:55:40,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-27 23:55:40,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-27 23:55:40,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-27 23:55:40,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-27 23:55:40,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-27 23:55:40,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-27 23:55:40,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-27 23:55:40,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-27 23:55:40,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-27 23:55:40,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-27 23:55:40,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-27 23:55:40,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-27 23:55:40,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-27 23:55:40,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-27 23:55:40,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-27 23:55:40,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-27 23:55:40,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-27 23:55:40,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-27 23:55:40,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-27 23:55:40,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-27 23:55:40,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-27 23:55:40,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-27 23:55:40,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-27 23:55:40,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-27 23:55:40,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-27 23:55:40,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-27 23:55:40,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-27 23:55:40,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-27 23:55:40,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-27 23:55:40,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-27 23:55:40,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-27 23:55:40,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-27 23:55:40,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-27 23:55:40,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-27 23:55:40,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-27 23:55:40,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-27 23:55:40,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-27 23:55:40,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-27 23:55:40,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-27 23:55:40,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-27 23:55:40,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-27 23:55:40,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-27 23:55:40,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-27 23:55:40,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-27 23:55:40,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-27 23:55:40,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-27 23:55:40,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-27 23:55:40,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-27 23:55:40,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-27 23:55:40,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-27 23:55:40,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-27 23:55:40,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-27 23:55:40,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-27 23:55:40,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-27 23:55:40,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-27 23:55:40,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-27 23:55:40,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-27 23:55:40,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-27 23:55:40,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-27 23:55:40,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-27 23:55:40,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-27 23:55:40,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-27 23:55:40,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-27 23:55:40,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-27 23:55:40,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-27 23:55:40,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-27 23:55:40,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-27 23:55:40,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-27 23:55:40,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-27 23:55:40,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-27 23:55:40,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-27 23:55:40,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-27 23:55:40,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-27 23:55:40,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-27 23:55:40,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-27 23:55:40,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-27 23:55:40,332 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039834737791963476, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05433198031820134, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039834737791963476, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17141666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23633333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-27 23:55:40,333 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-27 23:55:40,333 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17603534102359444, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039834737791963476, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05433198031820134, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039834737791963476, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17141666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23633333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-27 23:55:40,590 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 354 ms to try 5000 different iterations.  Found a solution that moves 2559 regions; Going from a computed imbalance of 0.17603534102359444 to a new imbalance of 0.01497757445555382. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.025094409590431802, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.036496390670674984, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2843333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.025094409590431802, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012083333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.042166666666666665, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-27 23:55:40,599 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1061286597:77 , srv2091865907:78 , srv1686802060:80 , srv1880024035:80 , srv861703114:80 , srv1635032696:81 , srv1921850322:81 , srv2048253829:81 , srv2772161:81 , srv1508387942:82 , srv323570204:82 , srv495097909:82 , srv498651332:82 , srv547045250:82 , srv757529721:82 , srv1575352750:83 , srv2059829393:83 , srv351723097:83 , srv40515242:83 , srv464779117:83 , srv1225818519:84 , srv1239681650:84 , srv1247334324:84 , srv128646128:84 , srv1515753254:84 , srv1563005908:84 , srv1636231149:84 , srv1797829109:84 , srv2030171466:84 , srv342024123:84 , srv508811828:84 , srv606120520:84 , srv641545053:84 , srv654847029:84 , srv747989542:84 , srv886991066:84 , srv987584409:84 , srv1052391350:85 , srv111259787:85 , srv1521400614:85 , srv1726768514:85 , srv1729353179:85 , srv1885391765:85 , srv1954474731:85 , srv1963901146:85 , srv2065772104:85 , srv304618238:85 , srv690312031:85 , srv813267045:85 , srv971362344:85 , srv107361962:86 , srv1109957176:86 , srv1160416620:86 , srv1421815941:86 , srv1548424272:86 , srv157956004:86 , srv1620935245:86 , srv1694326431:86 , srv197053933:86 , srv2068925692:86 , srv2089623367:86 , srv2106568965:86 , srv2137824743:86 , srv433535584:86 , srv459878999:86 , srv492505027:86 , srv661620231:86 , srv798549144:86 , srv884113348:86 , srv1134916488:87 , srv1311185486:87 , srv1630677763:87 , srv1891324134:87 , srv291264504:87 , srv307548559:87 , srv377425857:87 , srv936693421:87 , srv1146912502:88 , srv1658859122:88 , srv1674505315:88 , srv188408159:88 , srv417083115:88 , srv55108175:88 , srv682204729:88 , srv956690517:88 , srv96349630:88 , srv990101711:88 , srv143504573:89 , srv1892847137:89 , srv2078478244:89 , srv2106667467:89 , srv614293029:89 , srv1500632723:90 , srv1637925814:90 , srv1969224337:90 , srv759498598:90 , srv88104840:90 , srv1857037615:91 , srv775604873:92 , srv1993853609:553 }
2023-04-27 23:55:40,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-27 23:55:40,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1857037615=35, srv1954474731=42, srv2030171466=47, srv1575352750=21, srv1247334324=10, srv342024123=63, srv1637925814=27, srv1146912502=6, srv1921850322=41, srv886991066=93, srv1635032696=25, srv1694326431=31, srv2059829393=49, srv492505027=71, srv1620935245=23, srv304618238=60, srv1160416620=7, srv498651332=73, srv990101711=99, srv1797829109=34, srv188408159=37, srv2068925692=51, srv747989542=84, srv661620231=81, srv1421815941=13, srv323570204=62, srv1508387942=16, srv2091865907=54, srv641545053=79, srv307548559=61, srv1521400614=18, srv1636231149=26, srv936693421=94, srv1993853609=46, srv88104840=91, srv1061286597=1, srv157956004=22, srv606120520=77, srv861703114=90, srv55108175=76, srv377425857=65, srv614293029=78, srv495097909=72, srv1225818519=8, srv1134916488=5, srv1892847137=40, srv956690517=95, srv1548424272=19, srv1726768514=32, srv690312031=83, srv111259787=4, srv2089623367=53, srv654847029=80, srv1891324134=39, srv2137824743=57, srv1563005908=20, srv197053933=45, srv1963901146=43, srv987584409=98, srv1880024035=36, srv971362344=97, srv1515753254=17, srv757529721=85, srv759498598=86, srv798549144=88, srv1686802060=30, srv775604873=87, srv2078478244=52, srv1109957176=3, srv2106568965=55, srv1239681650=9, srv433535584=68, srv96349630=96, srv884113348=92, srv417083115=67, srv2772161=58, srv291264504=59, srv1052391350=0, srv40515242=66, srv351723097=64, srv2106667467=56, srv128646128=11, srv508811828=74, srv1500632723=15, srv2048253829=48, srv459878999=69, srv682204729=82, srv464779117=70, srv1969224337=44, srv547045250=75, srv1658859122=28, srv1674505315=29, srv2065772104=50, srv1729353179=33, srv1885391765=38, srv143504573=14, srv813267045=89, srv1311185486=12, srv107361962=2, srv1630677763=24} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-27 23:55:40,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-27 23:55:40,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-27 23:55:40,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-27 23:55:40,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-27 23:55:40,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-27 23:55:40,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-27 23:55:40,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-27 23:55:40,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-27 23:55:40,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-27 23:55:40,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-27 23:55:40,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-27 23:55:40,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-27 23:55:40,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-27 23:55:40,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-27 23:55:40,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-27 23:55:40,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-27 23:55:40,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-27 23:55:40,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-27 23:55:40,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-27 23:55:40,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-27 23:55:40,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-27 23:55:40,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-27 23:55:40,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-27 23:55:40,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-27 23:55:40,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-27 23:55:40,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-27 23:55:40,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-27 23:55:40,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-27 23:55:40,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-27 23:55:40,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-27 23:55:40,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-27 23:55:40,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-27 23:55:40,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-27 23:55:40,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-27 23:55:40,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-27 23:55:40,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-27 23:55:40,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-27 23:55:40,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-27 23:55:40,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-27 23:55:40,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-27 23:55:40,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-27 23:55:40,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-27 23:55:40,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-27 23:55:40,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-27 23:55:40,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-27 23:55:40,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-27 23:55:40,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-27 23:55:40,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-27 23:55:40,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-27 23:55:40,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-27 23:55:40,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-27 23:55:40,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-27 23:55:40,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-27 23:55:40,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-27 23:55:40,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-27 23:55:40,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-27 23:55:40,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-27 23:55:40,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-27 23:55:40,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-27 23:55:40,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-27 23:55:40,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-27 23:55:40,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-27 23:55:40,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-27 23:55:40,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-27 23:55:40,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-27 23:55:40,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-27 23:55:40,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-27 23:55:40,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-27 23:55:40,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-27 23:55:40,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-27 23:55:40,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-27 23:55:40,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-27 23:55:40,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-27 23:55:40,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-27 23:55:40,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-27 23:55:40,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-27 23:55:40,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-27 23:55:40,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-27 23:55:40,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-27 23:55:40,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-27 23:55:40,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-27 23:55:40,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-27 23:55:40,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-27 23:55:40,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-27 23:55:40,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-27 23:55:40,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-27 23:55:40,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-27 23:55:40,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-27 23:55:40,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-27 23:55:40,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-27 23:55:40,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-27 23:55:40,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-27 23:55:40,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-27 23:55:40,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-27 23:55:40,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-27 23:55:40,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-27 23:55:40,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-27 23:55:40,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-27 23:55:40,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-27 23:55:40,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-27 23:55:40,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-27 23:55:40,749 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.025094409590431802, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.036496390670674984, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.025094409590431802, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012083333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.042166666666666665, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-27 23:55:40,749 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-27 23:55:40,749 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.014959655954574018, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.025094409590431802, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.036496390670674984, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.025094409590431802, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012083333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.042166666666666665, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-27 23:55:41,022 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 368 ms to try 5000 different iterations.  Found a solution that moves 942 regions; Going from a computed imbalance of 0.014959655954574018 to a new imbalance of 8.826154384185966E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007772164079624705); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009628334541246444); MoveCostFunction : (multiplier=7.0, imbalance=0.10466666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007772164079624705); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004666666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-27 23:55:41,025 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv2065772104:81 , srv1620935245:82 , srv1892847137:82 , srv107361962:83 , srv1674505315:83 , srv1726768514:83 , srv417083115:83 , srv759498598:83 , srv775604873:83 , srv956690517:83 , srv111259787:84 , srv1421815941:84 , srv143504573:84 , srv1500632723:84 , srv1637925814:84 , srv1658859122:84 , srv1729353179:84 , srv188408159:84 , srv1954474731:84 , srv2091865907:84 , srv2106568965:84 , srv2106667467:84 , srv2137824743:84 , srv307548559:84 , srv495097909:84 , srv747989542:84 , srv813267045:84 , srv936693421:84 , srv987584409:84 , srv1146912502:85 , srv1247334324:85 , srv1311185486:85 , srv1521400614:85 , srv1548424272:85 , srv1630677763:85 , srv1636231149:85 , srv1694326431:85 , srv1857037615:85 , srv1880024035:85 , srv1885391765:85 , srv1921850322:85 , srv2078478244:85 , srv291264504:85 , srv342024123:85 , srv459878999:85 , srv464779117:85 , srv492505027:85 , srv614293029:85 , srv690312031:85 , srv96349630:85 , srv1052391350:86 , srv1109957176:86 , srv1225818519:86 , srv1239681650:86 , srv157956004:86 , srv1635032696:86 , srv1797829109:86 , srv1891324134:86 , srv1963901146:86 , srv2772161:86 , srv304618238:86 , srv351723097:86 , srv377425857:86 , srv40515242:86 , srv55108175:86 , srv682204729:86 , srv798549144:86 , srv88104840:86 , srv886991066:86 , srv1134916488:87 , srv1508387942:87 , srv1515753254:87 , srv1563005908:87 , srv1575352750:87 , srv1686802060:87 , srv2030171466:87 , srv2048253829:87 , srv323570204:87 , srv508811828:87 , srv641545053:87 , srv654847029:87 , srv884113348:87 , srv990101711:87 , srv1061286597:88 , srv1160416620:88 , srv128646128:88 , srv1969224337:88 , srv197053933:88 , srv2059829393:88 , srv2089623367:88 , srv433535584:88 , srv547045250:88 , srv606120520:88 , srv661620231:88 , srv861703114:88 , srv971362344:88 , srv2068925692:89 , srv498651332:89 , srv757529721:89 , srv1993853609:530 }
2023-04-27 23:55:41,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-27 23:55:41,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1857037615=35, srv1954474731=42, srv2030171466=47, srv1575352750=21, srv1247334324=10, srv342024123=63, srv1637925814=27, srv1146912502=6, srv1921850322=41, srv886991066=93, srv1635032696=25, srv1694326431=31, srv2059829393=49, srv492505027=71, srv1620935245=23, srv304618238=60, srv1160416620=7, srv498651332=73, srv990101711=99, srv1797829109=34, srv188408159=37, srv2068925692=51, srv747989542=84, srv661620231=81, srv1421815941=13, srv323570204=62, srv1508387942=16, srv2091865907=54, srv641545053=79, srv307548559=61, srv1521400614=18, srv1636231149=26, srv936693421=94, srv1993853609=46, srv88104840=91, srv1061286597=1, srv157956004=22, srv606120520=77, srv861703114=90, srv55108175=76, srv377425857=65, srv614293029=78, srv495097909=72, srv1225818519=8, srv1134916488=5, srv1892847137=40, srv956690517=95, srv1548424272=19, srv1726768514=32, srv690312031=83, srv111259787=4, srv2089623367=53, srv654847029=80, srv1891324134=39, srv2137824743=57, srv1563005908=20, srv197053933=45, srv1963901146=43, srv987584409=98, srv1880024035=36, srv971362344=97, srv1515753254=17, srv757529721=85, srv759498598=86, srv798549144=88, srv1686802060=30, srv775604873=87, srv2078478244=52, srv1109957176=3, srv2106568965=55, srv1239681650=9, srv433535584=68, srv96349630=96, srv884113348=92, srv417083115=67, srv2772161=58, srv291264504=59, srv1052391350=0, srv40515242=66, srv351723097=64, srv2106667467=56, srv128646128=11, srv508811828=74, srv1500632723=15, srv2048253829=48, srv459878999=69, srv682204729=82, srv464779117=70, srv1969224337=44, srv547045250=75, srv1658859122=28, srv1674505315=29, srv2065772104=50, srv1729353179=33, srv1885391765=38, srv143504573=14, srv813267045=89, srv1311185486=12, srv107361962=2, srv1630677763=24} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-27 23:55:41,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-27 23:55:41,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-27 23:55:41,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-27 23:55:41,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-27 23:55:41,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-27 23:55:41,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-27 23:55:41,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-27 23:55:41,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-27 23:55:41,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-27 23:55:41,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-27 23:55:41,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-27 23:55:41,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-27 23:55:41,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-27 23:55:41,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-27 23:55:41,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-27 23:55:41,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-27 23:55:41,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-27 23:55:41,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-27 23:55:41,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-27 23:55:41,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-27 23:55:41,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-27 23:55:41,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-27 23:55:41,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-27 23:55:41,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-27 23:55:41,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-27 23:55:41,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-27 23:55:41,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-27 23:55:41,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-27 23:55:41,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-27 23:55:41,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-27 23:55:41,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-27 23:55:41,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-27 23:55:41,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-27 23:55:41,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-27 23:55:41,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-27 23:55:41,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-27 23:55:41,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-27 23:55:41,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-27 23:55:41,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-27 23:55:41,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-27 23:55:41,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-27 23:55:41,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-27 23:55:41,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-27 23:55:41,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-27 23:55:41,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-27 23:55:41,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-27 23:55:41,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-27 23:55:41,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-27 23:55:41,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-27 23:55:41,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-27 23:55:41,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-27 23:55:41,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-27 23:55:41,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-27 23:55:41,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-27 23:55:41,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-27 23:55:41,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-27 23:55:41,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-27 23:55:41,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-27 23:55:41,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-27 23:55:41,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-27 23:55:41,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-27 23:55:41,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-27 23:55:41,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-27 23:55:41,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-27 23:55:41,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-27 23:55:41,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-27 23:55:41,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-27 23:55:41,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-27 23:55:41,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-27 23:55:41,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-27 23:55:41,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-27 23:55:41,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-27 23:55:41,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-27 23:55:41,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-27 23:55:41,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-27 23:55:41,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-27 23:55:41,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-27 23:55:41,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-27 23:55:41,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-27 23:55:41,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-27 23:55:41,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-27 23:55:41,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-27 23:55:41,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-27 23:55:41,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-27 23:55:41,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-27 23:55:41,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-27 23:55:41,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-27 23:55:41,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-27 23:55:41,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-27 23:55:41,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-27 23:55:41,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-27 23:55:41,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-27 23:55:41,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-27 23:55:41,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-27 23:55:41,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-27 23:55:41,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-27 23:55:41,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-27 23:55:41,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-27 23:55:41,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-27 23:55:41,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-27 23:55:41,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-27 23:55:41,168 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007772164079624705); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009628334541246444); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007772164079624705); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004666666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-27 23:55:41,168 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-27 23:55:41,169 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=8.760194134389277E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007772164079624705); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009628334541246444); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007772164079624705); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004666666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-27 23:55:41,435 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 362 ms to try 5000 different iterations.  Found a solution that moves 277 regions; Going from a computed imbalance of 8.760194134389277E-4 to a new imbalance of 2.5043051324913033E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.5313429231318235E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014213381090374027); MoveCostFunction : (multiplier=7.0, imbalance=0.03077777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.5313429231318235E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-27 23:55:41,437 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv143504573:83 , srv323570204:83 , srv433535584:83 , srv498651332:83 , srv1134916488:84 , srv1146912502:84 , srv1311185486:84 , srv1421815941:84 , srv1500632723:84 , srv1508387942:84 , srv1521400614:84 , srv1563005908:84 , srv1575352750:84 , srv1726768514:84 , srv1729353179:84 , srv188408159:84 , srv1969224337:84 , srv2030171466:84 , srv2059829393:84 , srv2068925692:84 , srv2137824743:84 , srv2772161:84 , srv307548559:84 , srv459878999:84 , srv547045250:84 , srv606120520:84 , srv614293029:84 , srv654847029:84 , srv757529721:84 , srv775604873:84 , srv813267045:84 , srv861703114:84 , srv884113348:84 , srv96349630:84 , srv990101711:84 , srv1052391350:85 , srv1061286597:85 , srv107361962:85 , srv1109957176:85 , srv111259787:85 , srv1160416620:85 , srv1247334324:85 , srv128646128:85 , srv1515753254:85 , srv1548424272:85 , srv157956004:85 , srv1620935245:85 , srv1630677763:85 , srv1635032696:85 , srv1636231149:85 , srv1637925814:85 , srv1658859122:85 , srv1674505315:85 , srv1686802060:85 , srv1694326431:85 , srv1857037615:85 , srv1880024035:85 , srv1885391765:85 , srv1891324134:85 , srv1892847137:85 , srv1921850322:85 , srv1954474731:85 , srv1963901146:85 , srv197053933:85 , srv2078478244:85 , srv2089623367:85 , srv2091865907:85 , srv2106568965:85 , srv291264504:85 , srv304618238:85 , srv342024123:85 , srv351723097:85 , srv40515242:85 , srv417083115:85 , srv464779117:85 , srv492505027:85 , srv495097909:85 , srv508811828:85 , srv55108175:85 , srv641545053:85 , srv661620231:85 , srv682204729:85 , srv690312031:85 , srv747989542:85 , srv759498598:85 , srv798549144:85 , srv88104840:85 , srv936693421:85 , srv971362344:85 , srv987584409:85 , srv1225818519:86 , srv1239681650:86 , srv1797829109:86 , srv2048253829:86 , srv2065772104:86 , srv2106667467:86 , srv377425857:86 , srv886991066:86 , srv956690517:86 , srv1993853609:615 }
2023-04-27 23:55:41,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-27 23:55:41,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1857037615=35, srv1954474731=42, srv2030171466=47, srv1575352750=21, srv1247334324=10, srv342024123=63, srv1637925814=27, srv1146912502=6, srv1921850322=41, srv886991066=93, srv1635032696=25, srv1694326431=31, srv2059829393=49, srv492505027=71, srv1620935245=23, srv304618238=60, srv1160416620=7, srv498651332=73, srv990101711=99, srv1797829109=34, srv188408159=37, srv2068925692=51, srv747989542=84, srv661620231=81, srv1421815941=13, srv323570204=62, srv1508387942=16, srv2091865907=54, srv641545053=79, srv307548559=61, srv1521400614=18, srv1636231149=26, srv936693421=94, srv1993853609=46, srv88104840=91, srv1061286597=1, srv157956004=22, srv606120520=77, srv861703114=90, srv55108175=76, srv377425857=65, srv614293029=78, srv495097909=72, srv1225818519=8, srv1134916488=5, srv1892847137=40, srv956690517=95, srv1548424272=19, srv1726768514=32, srv690312031=83, srv111259787=4, srv2089623367=53, srv654847029=80, srv1891324134=39, srv2137824743=57, srv1563005908=20, srv197053933=45, srv1963901146=43, srv987584409=98, srv1880024035=36, srv971362344=97, srv1515753254=17, srv757529721=85, srv759498598=86, srv798549144=88, srv1686802060=30, srv775604873=87, srv2078478244=52, srv1109957176=3, srv2106568965=55, srv1239681650=9, srv433535584=68, srv96349630=96, srv884113348=92, srv417083115=67, srv2772161=58, srv291264504=59, srv1052391350=0, srv40515242=66, srv351723097=64, srv2106667467=56, srv128646128=11, srv508811828=74, srv1500632723=15, srv2048253829=48, srv459878999=69, srv682204729=82, srv464779117=70, srv1969224337=44, srv547045250=75, srv1658859122=28, srv1674505315=29, srv2065772104=50, srv1729353179=33, srv1885391765=38, srv143504573=14, srv813267045=89, srv1311185486=12, srv107361962=2, srv1630677763=24} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-27 23:55:41,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-27 23:55:41,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-27 23:55:41,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-27 23:55:41,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-27 23:55:41,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-27 23:55:41,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-27 23:55:41,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-27 23:55:41,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-27 23:55:41,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-27 23:55:41,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-27 23:55:41,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-27 23:55:41,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-27 23:55:41,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-27 23:55:41,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-27 23:55:41,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-27 23:55:41,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-27 23:55:41,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-27 23:55:41,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-27 23:55:41,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-27 23:55:41,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-27 23:55:41,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-27 23:55:41,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-27 23:55:41,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-27 23:55:41,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-27 23:55:41,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-27 23:55:41,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-27 23:55:41,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-27 23:55:41,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-27 23:55:41,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-27 23:55:41,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-27 23:55:41,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-27 23:55:41,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-27 23:55:41,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-27 23:55:41,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-27 23:55:41,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-27 23:55:41,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-27 23:55:41,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-27 23:55:41,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-27 23:55:41,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-27 23:55:41,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-27 23:55:41,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-27 23:55:41,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-27 23:55:41,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-27 23:55:41,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-27 23:55:41,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-27 23:55:41,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-27 23:55:41,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-27 23:55:41,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-27 23:55:41,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-27 23:55:41,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-27 23:55:41,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-27 23:55:41,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-27 23:55:41,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-27 23:55:41,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-27 23:55:41,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-27 23:55:41,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-27 23:55:41,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-27 23:55:41,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-27 23:55:41,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-27 23:55:41,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-27 23:55:41,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-27 23:55:41,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-27 23:55:41,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-27 23:55:41,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-27 23:55:41,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-27 23:55:41,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-27 23:55:41,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-27 23:55:41,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-27 23:55:41,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-27 23:55:41,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-27 23:55:41,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-27 23:55:41,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-27 23:55:41,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-27 23:55:41,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-27 23:55:41,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-27 23:55:41,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-27 23:55:41,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-27 23:55:41,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-27 23:55:41,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-27 23:55:41,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-27 23:55:41,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-27 23:55:41,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-27 23:55:41,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-27 23:55:41,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-27 23:55:41,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-27 23:55:41,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-27 23:55:41,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-27 23:55:41,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-27 23:55:41,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-27 23:55:41,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-27 23:55:41,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-27 23:55:41,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-27 23:55:41,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-27 23:55:41,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-27 23:55:41,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-27 23:55:41,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-27 23:55:41,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-27 23:55:41,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-27 23:55:41,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-27 23:55:41,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-27 23:55:41,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-27 23:55:41,560 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.5313429231318235E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014213381090374027); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.5313429231318235E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-27 23:55:41,560 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-27 23:55:41,560 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=2.31034558691081E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.5313429231318235E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014213381090374027); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.5313429231318235E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-27 23:55:41,777 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 309 ms to try 5000 different iterations.  Found a solution that moves 11 regions; Going from a computed imbalance of 2.31034558691081E-5 to a new imbalance of 6.10910238598777E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0013400504203456161); MoveCostFunction : (multiplier=7.0, imbalance=0.0012222222222222222); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-27 23:55:41,778 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1239681650:83 , srv2048253829:83 , srv2106667467:83 , srv798549144:83 , srv886991066:83 , srv1052391350:84 , srv1061286597:84 , srv107361962:84 , srv1109957176:84 , srv111259787:84 , srv1134916488:84 , srv1160416620:84 , srv1225818519:84 , srv1247334324:84 , srv128646128:84 , srv1311185486:84 , srv1421815941:84 , srv1500632723:84 , srv1508387942:84 , srv1515753254:84 , srv1521400614:84 , srv1548424272:84 , srv1563005908:84 , srv1575352750:84 , srv157956004:84 , srv1620935245:84 , srv1630677763:84 , srv1635032696:84 , srv1636231149:84 , srv1637925814:84 , srv1658859122:84 , srv1674505315:84 , srv1686802060:84 , srv1694326431:84 , srv1726768514:84 , srv1797829109:84 , srv1857037615:84 , srv1880024035:84 , srv188408159:84 , srv1885391765:84 , srv1891324134:84 , srv1892847137:84 , srv1921850322:84 , srv1954474731:84 , srv1963901146:84 , srv1969224337:84 , srv197053933:84 , srv2030171466:84 , srv2059829393:84 , srv2065772104:84 , srv2068925692:84 , srv2078478244:84 , srv2089623367:84 , srv2091865907:84 , srv2106568965:84 , srv2137824743:84 , srv2772161:84 , srv291264504:84 , srv304618238:84 , srv307548559:84 , srv342024123:84 , srv351723097:84 , srv377425857:84 , srv40515242:84 , srv417083115:84 , srv433535584:84 , srv459878999:84 , srv464779117:84 , srv492505027:84 , srv495097909:84 , srv498651332:84 , srv508811828:84 , srv547045250:84 , srv55108175:84 , srv606120520:84 , srv614293029:84 , srv641545053:84 , srv654847029:84 , srv661620231:84 , srv682204729:84 , srv690312031:84 , srv747989542:84 , srv757529721:84 , srv759498598:84 , srv775604873:84 , srv813267045:84 , srv88104840:84 , srv884113348:84 , srv936693421:84 , srv956690517:84 , srv96349630:84 , srv971362344:84 , srv987584409:84 , srv990101711:84 , srv1146912502:85 , srv143504573:85 , srv1729353179:85 , srv323570204:85 , srv861703114:85 , srv1993853609:684 }
2023-04-27 23:55:41,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-27 23:55:41,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1857037615=35, srv1954474731=42, srv2030171466=47, srv1575352750=21, srv1247334324=10, srv342024123=63, srv1637925814=27, srv1146912502=6, srv1921850322=41, srv886991066=93, srv1635032696=25, srv1694326431=31, srv2059829393=49, srv492505027=71, srv1620935245=23, srv304618238=60, srv1160416620=7, srv498651332=73, srv990101711=99, srv1797829109=34, srv188408159=37, srv2068925692=51, srv747989542=84, srv661620231=81, srv1421815941=13, srv323570204=62, srv1508387942=16, srv2091865907=54, srv641545053=79, srv307548559=61, srv1521400614=18, srv1636231149=26, srv936693421=94, srv1993853609=46, srv88104840=91, srv1061286597=1, srv157956004=22, srv606120520=77, srv861703114=90, srv55108175=76, srv377425857=65, srv614293029=78, srv495097909=72, srv1225818519=8, srv1134916488=5, srv1892847137=40, srv956690517=95, srv1548424272=19, srv1726768514=32, srv690312031=83, srv111259787=4, srv2089623367=53, srv654847029=80, srv1891324134=39, srv2137824743=57, srv1563005908=20, srv197053933=45, srv1963901146=43, srv987584409=98, srv1880024035=36, srv971362344=97, srv1515753254=17, srv757529721=85, srv759498598=86, srv798549144=88, srv1686802060=30, srv775604873=87, srv2078478244=52, srv1109957176=3, srv2106568965=55, srv1239681650=9, srv433535584=68, srv96349630=96, srv884113348=92, srv417083115=67, srv2772161=58, srv291264504=59, srv1052391350=0, srv40515242=66, srv351723097=64, srv2106667467=56, srv128646128=11, srv508811828=74, srv1500632723=15, srv2048253829=48, srv459878999=69, srv682204729=82, srv464779117=70, srv1969224337=44, srv547045250=75, srv1658859122=28, srv1674505315=29, srv2065772104=50, srv1729353179=33, srv1885391765=38, srv143504573=14, srv813267045=89, srv1311185486=12, srv107361962=2, srv1630677763=24} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-27 23:55:41,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-27 23:55:41,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-27 23:55:41,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-27 23:55:41,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-27 23:55:41,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-27 23:55:41,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-27 23:55:41,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-27 23:55:41,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-27 23:55:41,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-27 23:55:41,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-27 23:55:41,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-27 23:55:41,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-27 23:55:41,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-27 23:55:41,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-27 23:55:41,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-27 23:55:41,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-27 23:55:41,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-27 23:55:41,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-27 23:55:41,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-27 23:55:41,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-27 23:55:41,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-27 23:55:41,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-27 23:55:41,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-27 23:55:41,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-27 23:55:41,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-27 23:55:41,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-27 23:55:41,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-27 23:55:41,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-27 23:55:41,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-27 23:55:41,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-27 23:55:41,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-27 23:55:41,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-27 23:55:41,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-27 23:55:41,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-27 23:55:41,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-27 23:55:41,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-27 23:55:41,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-27 23:55:41,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-27 23:55:41,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-27 23:55:41,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-27 23:55:41,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-27 23:55:41,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-27 23:55:41,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-27 23:55:41,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-27 23:55:41,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-27 23:55:41,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-27 23:55:41,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-27 23:55:41,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-27 23:55:41,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-27 23:55:41,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-27 23:55:41,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-27 23:55:41,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-27 23:55:41,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-27 23:55:41,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-27 23:55:41,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-27 23:55:41,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-27 23:55:41,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-27 23:55:41,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-27 23:55:41,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-27 23:55:41,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-27 23:55:41,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-27 23:55:41,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-27 23:55:41,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-27 23:55:41,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-27 23:55:41,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-27 23:55:41,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-27 23:55:41,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-27 23:55:41,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-27 23:55:41,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-27 23:55:41,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-27 23:55:41,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-27 23:55:41,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-27 23:55:41,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-27 23:55:41,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-27 23:55:41,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-27 23:55:41,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-27 23:55:41,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-27 23:55:41,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-27 23:55:41,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-27 23:55:41,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-27 23:55:41,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-27 23:55:41,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-27 23:55:41,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-27 23:55:41,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-27 23:55:41,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-27 23:55:41,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-27 23:55:41,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-27 23:55:41,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-27 23:55:41,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-27 23:55:41,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-27 23:55:41,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-27 23:55:41,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-27 23:55:41,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-27 23:55:41,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-27 23:55:41,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-27 23:55:41,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-27 23:55:41,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-27 23:55:41,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-27 23:55:41,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-27 23:55:41,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-27 23:55:41,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-27 23:55:41,914 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=6.032078739728369E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0013400504203456161); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-27 23:55:41,915 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1239681650:83 , srv2048253829:83 , srv2106667467:83 , srv798549144:83 , srv886991066:83 , srv1052391350:84 , srv1061286597:84 , srv107361962:84 , srv1109957176:84 , srv111259787:84 , srv1134916488:84 , srv1160416620:84 , srv1225818519:84 , srv1247334324:84 , srv128646128:84 , srv1311185486:84 , srv1421815941:84 , srv1500632723:84 , srv1508387942:84 , srv1515753254:84 , srv1521400614:84 , srv1548424272:84 , srv1563005908:84 , srv1575352750:84 , srv157956004:84 , srv1620935245:84 , srv1630677763:84 , srv1635032696:84 , srv1636231149:84 , srv1637925814:84 , srv1658859122:84 , srv1674505315:84 , srv1686802060:84 , srv1694326431:84 , srv1726768514:84 , srv1797829109:84 , srv1857037615:84 , srv1880024035:84 , srv188408159:84 , srv1885391765:84 , srv1891324134:84 , srv1892847137:84 , srv1921850322:84 , srv1954474731:84 , srv1963901146:84 , srv1969224337:84 , srv197053933:84 , srv2030171466:84 , srv2059829393:84 , srv2065772104:84 , srv2068925692:84 , srv2078478244:84 , srv2089623367:84 , srv2091865907:84 , srv2106568965:84 , srv2137824743:84 , srv2772161:84 , srv291264504:84 , srv304618238:84 , srv307548559:84 , srv342024123:84 , srv351723097:84 , srv377425857:84 , srv40515242:84 , srv417083115:84 , srv433535584:84 , srv459878999:84 , srv464779117:84 , srv492505027:84 , srv495097909:84 , srv498651332:84 , srv508811828:84 , srv547045250:84 , srv55108175:84 , srv606120520:84 , srv614293029:84 , srv641545053:84 , srv654847029:84 , srv661620231:84 , srv682204729:84 , srv690312031:84 , srv747989542:84 , srv757529721:84 , srv759498598:84 , srv775604873:84 , srv813267045:84 , srv88104840:84 , srv884113348:84 , srv936693421:84 , srv956690517:84 , srv96349630:84 , srv971362344:84 , srv987584409:84 , srv990101711:84 , srv1146912502:85 , srv143504573:85 , srv1729353179:85 , srv323570204:85 , srv861703114:85 , srv1993853609:684 }
2023-04-27 23:55:41,939 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=437 (was 437), ProcessCount=171 (was 171), AvailableMemoryMB=8892 (was 9394)
2023-04-27 23:55:41,947 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=437, ProcessCount=171, AvailableMemoryMB=8891
2023-04-27 23:55:41,947 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-27 23:55:41,947 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-04-27 23:55:41,955 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1059122749:84 , srv1091361971:84 , srv1123317896:84 , srv1144021244:84 , srv1206047512:84 , srv1470020032:84 , srv1497911110:84 , srv1508323120:84 , srv1604866603:84 , srv1694049906:84 , srv1758674424:84 , srv1765896578:84 , srv1958626771:84 , srv1972099451:84 , srv204598895:84 , srv2083713049:84 , srv2090729492:84 , srv231859340:84 , srv365324715:84 , srv457790645:84 , srv504002586:84 , srv53942533:84 , srv581798374:84 , srv652281285:84 , srv715143921:84 , srv78989465:84 , srv862561774:84 , srv937622227:84 , srv946187994:84 , srv1620019009:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-04-27 23:55:41,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-27 23:55:41,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1620019009=9, srv1123317896=2, srv1091361971=1, srv581798374=23, srv715143921=25, srv1694049906=10, srv2090729492=17, srv365324715=19, srv937622227=28, srv1508323120=7, srv231859340=18, srv1972099451=14, srv1206047512=4, srv204598895=15, srv946187994=29, srv1604866603=8, srv652281285=24, srv53942533=22, srv1758674424=11, srv1765896578=12, srv457790645=20, srv504002586=21, srv862561774=27, srv1144021244=3, srv1497911110=6, srv1958626771=13, srv1470020032=5, srv1059122749=0, srv78989465=26, srv2083713049=16} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-27 23:55:41,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-27 23:55:41,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-27 23:55:41,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-27 23:55:41,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-27 23:55:41,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-27 23:55:41,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-27 23:55:41,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-27 23:55:41,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-27 23:55:41,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-27 23:55:41,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-27 23:55:41,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-27 23:55:41,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-27 23:55:41,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-27 23:55:41,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-27 23:55:41,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-27 23:55:41,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-27 23:55:41,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-27 23:55:41,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-27 23:55:41,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-27 23:55:41,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-27 23:55:41,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-27 23:55:41,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-27 23:55:41,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-27 23:55:41,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-27 23:55:41,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-27 23:55:41,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-27 23:55:41,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-27 23:55:41,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-27 23:55:41,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-27 23:55:41,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-27 23:55:41,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-04-27 23:55:41,981 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.089347654515639, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-27 23:55:41,982 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912474574806189, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.089347654515639, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-04-27 23:57:41,970 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6307553 different iterations.  Found a solution that moves 1838 regions; Going from a computed imbalance of 0.9912474574806189 to a new imbalance of 4.2899836916600065E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6807407407407408, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-27 23:57:41,977 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1059122749:90 , srv1091361971:90 , srv1123317896:90 , srv1144021244:90 , srv1206047512:90 , srv1470020032:90 , srv1497911110:90 , srv1508323120:90 , srv1604866603:90 , srv1620019009:90 , srv1694049906:90 , srv1758674424:90 , srv1765896578:90 , srv1958626771:90 , srv1972099451:90 , srv204598895:90 , srv2083713049:90 , srv2090729492:90 , srv231859340:90 , srv365324715:90 , srv457790645:90 , srv504002586:90 , srv53942533:90 , srv581798374:90 , srv652281285:90 , srv715143921:90 , srv78989465:90 , srv862561774:90 , srv937622227:90 , srv946187994:90 }
2023-04-27 23:57:41,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-27 23:57:41,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1620019009=9, srv1123317896=2, srv1091361971=1, srv581798374=23, srv715143921=25, srv1694049906=10, srv2090729492=17, srv365324715=19, srv937622227=28, srv1508323120=7, srv231859340=18, srv1972099451=14, srv1206047512=4, srv204598895=15, srv946187994=29, srv1604866603=8, srv652281285=24, srv53942533=22, srv1758674424=11, srv1765896578=12, srv457790645=20, srv504002586=21, srv862561774=27, srv1144021244=3, srv1497911110=6, srv1958626771=13, srv1470020032=5, srv1059122749=0, srv78989465=26, srv2083713049=16} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-27 23:57:41,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-27 23:57:41,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-27 23:57:41,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-27 23:57:41,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-27 23:57:41,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-27 23:57:41,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-27 23:57:41,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-27 23:57:41,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-27 23:57:41,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-27 23:57:41,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-27 23:57:41,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-27 23:57:41,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-27 23:57:41,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-27 23:57:41,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-27 23:57:41,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-27 23:57:41,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-27 23:57:41,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-27 23:57:41,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-27 23:57:41,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-27 23:57:41,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-27 23:57:41,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-27 23:57:41,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-27 23:57:41,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-27 23:57:41,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-27 23:57:41,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-27 23:57:41,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-27 23:57:41,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-27 23:57:41,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-27 23:57:41,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-27 23:57:41,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-27 23:57:41,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-04-27 23:57:41,998 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-27 23:57:41,998 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1059122749:90 , srv1091361971:90 , srv1123317896:90 , srv1144021244:90 , srv1206047512:90 , srv1470020032:90 , srv1497911110:90 , srv1508323120:90 , srv1604866603:90 , srv1620019009:90 , srv1694049906:90 , srv1758674424:90 , srv1765896578:90 , srv1958626771:90 , srv1972099451:90 , srv204598895:90 , srv2083713049:90 , srv2090729492:90 , srv231859340:90 , srv365324715:90 , srv457790645:90 , srv504002586:90 , srv53942533:90 , srv581798374:90 , srv652281285:90 , srv715143921:90 , srv78989465:90 , srv862561774:90 , srv937622227:90 , srv946187994:90 }
2023-04-27 23:57:42,010 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=411 (was 437), ProcessCount=172 (was 171) - ProcessCount LEAK? -, AvailableMemoryMB=8712 (was 8891)