2023-06-06 09:53:22,673 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-06-06 09:53:22,951 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-06-06 09:53:23,053 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-06-06 09:53:23,107 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-06-06 09:53:23,113 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-06-06 09:53:23,126 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=280, ProcessCount=176, AvailableMemoryMB=7981
2023-06-06 09:53:23,128 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-06-06 09:53:23,128 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-06-06 09:53:23,262 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1001219384:84 , srv1011111433:84 , srv1072728573:84 , srv1087664677:84 , srv1103063586:84 , srv1103687522:84 , srv1123604432:84 , srv1147301365:84 , srv115184015:84 , srv1162526869:84 , srv1166043025:84 , srv1169997071:84 , srv1211532695:84 , srv1219674725:84 , srv126007703:84 , srv1324254384:84 , srv1346214138:84 , srv1363644928:84 , srv1376349775:84 , srv1397335643:84 , srv1402055669:84 , srv1405423579:84 , srv1420630196:84 , srv1421618701:84 , srv1421918462:84 , srv1431708720:84 , srv1469921972:84 , srv1473496604:84 , srv1482914116:84 , srv1501337522:84 , srv1521273036:84 , srv1523554891:84 , srv153168181:84 , srv1584348994:84 , srv1607496207:84 , srv1635990727:84 , srv1637402067:84 , srv1658879280:84 , srv1671679015:84 , srv1675417981:84 , srv1717256805:84 , srv1757449747:84 , srv1759998728:84 , srv1773088374:84 , srv1783376095:84 , srv186642537:84 , srv1882312624:84 , srv190361647:84 , srv1918629159:84 , srv2001549620:84 , srv2008370346:84 , srv2062088207:84 , srv2074984594:84 , srv217357422:84 , srv217728533:84 , srv218172699:84 , srv229931174:84 , srv232501967:84 , srv235120598:84 , srv239371293:84 , srv239923164:84 , srv252048947:84 , srv263244108:84 , srv280449050:84 , srv292432957:84 , srv293287682:84 , srv319521401:84 , srv326956050:84 , srv368534157:84 , srv391843454:84 , srv404741158:84 , srv430728535:84 , srv432524758:84 , srv448967789:84 , srv489120961:84 , srv498012471:84 , srv538485867:84 , srv558011722:84 , srv608079460:84 , srv613546467:84 , srv619633958:84 , srv630966934:84 , srv691737124:84 , srv695494231:84 , srv69952195:84 , srv715310236:84 , srv757486050:84 , srv76037377:84 , srv763565468:84 , srv785690480:84 , srv812422662:84 , srv816302714:84 , srv823425825:84 , srv857813278:84 , srv873296832:84 , srv89397603:84 , srv924967326:84 , srv942724217:84 , srv95057042:84 , srv1927897661:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-06-06 09:53:23,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-06-06 09:53:23,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv319521401=67, srv292432957=65, srv89397603=96, srv1717256805=40, srv1166043025=10, srv1635990727=35, srv1637402067=36, srv252048947=62, srv1363644928=17, srv430728535=72, srv95057042=99, srv217728533=55, srv812422662=91, srv263244108=63, srv1759998728=42, srv873296832=95, srv1773088374=43, srv1211532695=12, srv538485867=77, srv391843454=70, srv239923164=61, srv1405423579=21, srv695494231=84, srv69952195=85, srv1123604432=6, srv432524758=73, srv239371293=60, srv715310236=86, srv1001219384=0, srv293287682=66, srv691737124=83, srv153168181=32, srv1469921972=26, srv608079460=79, srv1501337522=29, srv235120598=59, srv1376349775=18, srv1147301365=7, srv619633958=81, srv1421918462=24, srv1675417981=39, srv76037377=88, srv1607496207=34, srv816302714=92, srv924967326=97, srv1473496604=27, srv404741158=71, srv217357422=54, srv280449050=64, srv229931174=57, srv1162526869=9, srv1421618701=23, srv498012471=76, srv1087664677=3, srv1658879280=37, srv218172699=56, srv1521273036=30, srv126007703=14, srv1584348994=33, srv1011111433=1, srv1103687522=5, srv368534157=69, srv613546467=80, srv1103063586=4, srv1918629159=48, srv1169997071=11, srv115184015=8, srv823425825=93, srv326956050=68, srv2062088207=52, srv2074984594=53, srv757486050=87, srv1397335643=19, srv2001549620=50, srv1420630196=22, srv1072728573=2, srv1219674725=13, srv1783376095=44, srv186642537=45, srv942724217=98, srv1324254384=15, srv558011722=78, srv1346214138=16, srv190361647=47, srv448967789=74, srv1757449747=41, srv763565468=89, srv1882312624=46, srv1482914116=28, srv1431708720=25, srv1927897661=49, srv857813278=94, srv489120961=75, srv232501967=58, srv785690480=90, srv1671679015=38, srv2008370346=51, srv1523554891=31, srv630966934=82, srv1402055669=20} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-06-06 09:53:23,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-06-06 09:53:23,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-06-06 09:53:23,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-06-06 09:53:23,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-06-06 09:53:23,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-06-06 09:53:23,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-06-06 09:53:23,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-06-06 09:53:23,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-06-06 09:53:23,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-06-06 09:53:23,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-06-06 09:53:23,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-06-06 09:53:23,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-06-06 09:53:23,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-06-06 09:53:23,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-06-06 09:53:23,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-06-06 09:53:23,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-06-06 09:53:23,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-06-06 09:53:23,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-06-06 09:53:23,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-06-06 09:53:23,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-06-06 09:53:23,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-06-06 09:53:23,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-06-06 09:53:23,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-06-06 09:53:23,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-06-06 09:53:23,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-06-06 09:53:23,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-06-06 09:53:23,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-06-06 09:53:23,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-06-06 09:53:23,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-06-06 09:53:23,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-06-06 09:53:23,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-06-06 09:53:23,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-06-06 09:53:23,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-06-06 09:53:23,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-06-06 09:53:23,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-06-06 09:53:23,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-06-06 09:53:23,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-06-06 09:53:23,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-06-06 09:53:23,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-06-06 09:53:23,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-06-06 09:53:23,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-06-06 09:53:23,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-06-06 09:53:23,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-06-06 09:53:23,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-06-06 09:53:23,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-06-06 09:53:23,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-06-06 09:53:23,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-06-06 09:53:23,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-06-06 09:53:23,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-06-06 09:53:23,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-06-06 09:53:23,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-06-06 09:53:23,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-06-06 09:53:23,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-06-06 09:53:23,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-06-06 09:53:23,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-06-06 09:53:23,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-06-06 09:53:23,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-06-06 09:53:23,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-06-06 09:53:23,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-06-06 09:53:23,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-06-06 09:53:23,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-06-06 09:53:23,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-06-06 09:53:23,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-06-06 09:53:23,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-06-06 09:53:23,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-06-06 09:53:23,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-06-06 09:53:23,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-06-06 09:53:23,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-06-06 09:53:23,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-06-06 09:53:23,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-06-06 09:53:23,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-06-06 09:53:23,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-06-06 09:53:23,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-06-06 09:53:23,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-06-06 09:53:23,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-06-06 09:53:23,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-06-06 09:53:23,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-06-06 09:53:23,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-06-06 09:53:23,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-06-06 09:53:23,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-06-06 09:53:23,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-06-06 09:53:23,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-06-06 09:53:23,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-06-06 09:53:23,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-06-06 09:53:23,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-06-06 09:53:23,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-06-06 09:53:23,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-06-06 09:53:23,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-06-06 09:53:23,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-06-06 09:53:23,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-06-06 09:53:23,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-06-06 09:53:23,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-06-06 09:53:23,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-06-06 09:53:23,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-06-06 09:53:23,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-06-06 09:53:23,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-06-06 09:53:23,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-06-06 09:53:23,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-06-06 09:53:23,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-06-06 09:53:23,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-06-06 09:53:23,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-06-06 09:53:23,565 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-06-06 09:53:23,566 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-06-06 09:53:23,566 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-06-06 09:53:23,863 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 470 ms to try 5000 different iterations.  Found a solution that moves 4690 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1756207893553432. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04162043825344741, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05969808062683001, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5211111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04162043825344741, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17091666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.236, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-06-06 09:53:23,874 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv239371293:82 , srv391843454:82 , srv69952195:82 , srv812422662:82 , srv1087664677:83 , srv126007703:83 , srv1501337522:83 , srv1521273036:83 , srv153168181:83 , srv1635990727:83 , srv263244108:83 , srv319521401:83 , srv558011722:83 , srv608079460:83 , srv816302714:83 , srv1123604432:84 , srv1162526869:84 , srv1211532695:84 , srv1584348994:84 , srv1658879280:84 , srv1773088374:84 , srv1882312624:84 , srv2001549620:84 , srv235120598:84 , srv619633958:84 , srv695494231:84 , srv942724217:84 , srv1402055669:85 , srv1421618701:85 , srv1473496604:85 , srv232501967:85 , srv252048947:85 , srv280449050:85 , srv489120961:85 , srv498012471:85 , srv630966934:85 , srv691737124:85 , srv715310236:85 , srv757486050:85 , srv785690480:85 , srv89397603:85 , srv95057042:85 , srv1147301365:86 , srv115184015:86 , srv1397335643:86 , srv1420630196:86 , srv1607496207:86 , srv1759998728:86 , srv190361647:86 , srv217728533:86 , srv218172699:86 , srv229931174:86 , srv292432957:86 , srv538485867:86 , srv1072728573:87 , srv1431708720:87 , srv1675417981:87 , srv1757449747:87 , srv1783376095:87 , srv1918629159:87 , srv2008370346:87 , srv326956050:87 , srv368534157:87 , srv404741158:87 , srv448967789:87 , srv76037377:87 , srv857813278:87 , srv1001219384:88 , srv1166043025:88 , srv1169997071:88 , srv1346214138:88 , srv1376349775:88 , srv1421918462:88 , srv1469921972:88 , srv1523554891:88 , srv1637402067:88 , srv186642537:88 , srv2062088207:88 , srv217357422:88 , srv239923164:88 , srv430728535:88 , srv873296832:88 , srv1103063586:89 , srv1405423579:89 , srv1671679015:89 , srv1717256805:89 , srv293287682:89 , srv432524758:89 , srv823425825:89 , srv1219674725:90 , srv1482914116:90 , srv2074984594:90 , srv763565468:90 , srv924967326:90 , srv1011111433:91 , srv1324254384:91 , srv1363644928:91 , srv613546467:92 , srv1103687522:95 , srv1927897661:460 }
2023-06-06 09:53:23,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-06-06 09:53:23,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv319521401=67, srv292432957=65, srv89397603=96, srv1717256805=40, srv1166043025=10, srv1635990727=35, srv1637402067=36, srv252048947=62, srv1363644928=17, srv430728535=72, srv95057042=99, srv217728533=55, srv812422662=91, srv263244108=63, srv1759998728=42, srv873296832=95, srv1773088374=43, srv1211532695=12, srv538485867=77, srv391843454=70, srv239923164=61, srv1405423579=21, srv695494231=84, srv69952195=85, srv1123604432=6, srv432524758=73, srv239371293=60, srv715310236=86, srv1001219384=0, srv293287682=66, srv691737124=83, srv153168181=32, srv1469921972=26, srv608079460=79, srv1501337522=29, srv235120598=59, srv1376349775=18, srv1147301365=7, srv619633958=81, srv1421918462=24, srv1675417981=39, srv76037377=88, srv1607496207=34, srv816302714=92, srv924967326=97, srv1473496604=27, srv404741158=71, srv217357422=54, srv280449050=64, srv229931174=57, srv1162526869=9, srv1421618701=23, srv498012471=76, srv1087664677=3, srv1658879280=37, srv218172699=56, srv1521273036=30, srv126007703=14, srv1584348994=33, srv1011111433=1, srv1103687522=5, srv368534157=69, srv613546467=80, srv1103063586=4, srv1918629159=48, srv1169997071=11, srv115184015=8, srv823425825=93, srv326956050=68, srv2062088207=52, srv2074984594=53, srv757486050=87, srv1397335643=19, srv2001549620=50, srv1420630196=22, srv1072728573=2, srv1219674725=13, srv1783376095=44, srv186642537=45, srv942724217=98, srv1324254384=15, srv558011722=78, srv1346214138=16, srv190361647=47, srv448967789=74, srv1757449747=41, srv763565468=89, srv1882312624=46, srv1482914116=28, srv1431708720=25, srv1927897661=49, srv857813278=94, srv489120961=75, srv232501967=58, srv785690480=90, srv1671679015=38, srv2008370346=51, srv1523554891=31, srv630966934=82, srv1402055669=20} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-06-06 09:53:23,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-06-06 09:53:23,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-06-06 09:53:23,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-06-06 09:53:23,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-06-06 09:53:23,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-06-06 09:53:23,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-06-06 09:53:23,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-06-06 09:53:23,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-06-06 09:53:23,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-06-06 09:53:23,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-06-06 09:53:23,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-06-06 09:53:23,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-06-06 09:53:23,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-06-06 09:53:23,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-06-06 09:53:23,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-06-06 09:53:23,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-06-06 09:53:23,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-06-06 09:53:23,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-06-06 09:53:23,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-06-06 09:53:23,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-06-06 09:53:23,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-06-06 09:53:23,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-06-06 09:53:23,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-06-06 09:53:23,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-06-06 09:53:23,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-06-06 09:53:23,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-06-06 09:53:23,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-06-06 09:53:23,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-06-06 09:53:23,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-06-06 09:53:23,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-06-06 09:53:23,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-06-06 09:53:23,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-06-06 09:53:23,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-06-06 09:53:23,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-06-06 09:53:23,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-06-06 09:53:23,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-06-06 09:53:23,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-06-06 09:53:23,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-06-06 09:53:23,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-06-06 09:53:23,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-06-06 09:53:23,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-06-06 09:53:23,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-06-06 09:53:23,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-06-06 09:53:23,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-06-06 09:53:23,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-06-06 09:53:23,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-06-06 09:53:23,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-06-06 09:53:23,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-06-06 09:53:23,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-06-06 09:53:23,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-06-06 09:53:23,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-06-06 09:53:23,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-06-06 09:53:23,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-06-06 09:53:23,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-06-06 09:53:23,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-06-06 09:53:23,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-06-06 09:53:23,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-06-06 09:53:23,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-06-06 09:53:23,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-06-06 09:53:23,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-06-06 09:53:23,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-06-06 09:53:23,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-06-06 09:53:23,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-06-06 09:53:23,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-06-06 09:53:23,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-06-06 09:53:23,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-06-06 09:53:23,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-06-06 09:53:23,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-06-06 09:53:23,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-06-06 09:53:23,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-06-06 09:53:23,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-06-06 09:53:23,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-06-06 09:53:23,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-06-06 09:53:23,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-06-06 09:53:23,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-06-06 09:53:23,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-06-06 09:53:23,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-06-06 09:53:23,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-06-06 09:53:23,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-06-06 09:53:23,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-06-06 09:53:23,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-06-06 09:53:23,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-06-06 09:53:23,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-06-06 09:53:23,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-06-06 09:53:23,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-06-06 09:53:23,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-06-06 09:53:23,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-06-06 09:53:23,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-06-06 09:53:23,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-06-06 09:53:23,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-06-06 09:53:23,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-06-06 09:53:23,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-06-06 09:53:23,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-06-06 09:53:23,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-06-06 09:53:23,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-06-06 09:53:23,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-06-06 09:53:23,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-06-06 09:53:23,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-06-06 09:53:23,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-06-06 09:53:23,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-06-06 09:53:23,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-06-06 09:53:24,037 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04162043825344741, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05969808062683001, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04162043825344741, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17091666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.236, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-06-06 09:53:24,037 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-06-06 09:53:24,037 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17558794927343802, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04162043825344741, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05969808062683001, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04162043825344741, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17091666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.236, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-06-06 09:53:24,338 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 409 ms to try 5000 different iterations.  Found a solution that moves 2639 regions; Going from a computed imbalance of 0.17558794927343802 to a new imbalance of 0.01551370253254806. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022397178843598552); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03502764660858658, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2932222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022397178843598552); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01225); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04666666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-06-06 09:53:24,352 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1103687522:79 , srv1363644928:80 , srv1011111433:81 , srv1324254384:81 , srv293287682:81 , srv613546467:81 , srv1219674725:82 , srv1405423579:82 , srv186642537:82 , srv2074984594:82 , srv432524758:82 , srv763565468:82 , srv823425825:82 , srv924967326:82 , srv1001219384:83 , srv1072728573:83 , srv1169997071:83 , srv1482914116:83 , srv1523554891:83 , srv1675417981:83 , srv239923164:83 , srv76037377:83 , srv857813278:83 , srv873296832:83 , srv1103063586:84 , srv1420630196:84 , srv1421918462:84 , srv1431708720:84 , srv1469921972:84 , srv1637402067:84 , srv1671679015:84 , srv1717256805:84 , srv1757449747:84 , srv190361647:84 , srv2062088207:84 , srv217357422:84 , srv326956050:84 , srv448967789:84 , srv1147301365:85 , srv1346214138:85 , srv1397335643:85 , srv1402055669:85 , srv404741158:85 , srv115184015:86 , srv1166043025:86 , srv1376349775:86 , srv1584348994:86 , srv1607496207:86 , srv1783376095:86 , srv1882312624:86 , srv217728533:86 , srv218172699:86 , srv229931174:86 , srv368534157:86 , srv489120961:86 , srv942724217:86 , srv1421618701:87 , srv1473496604:87 , srv1658879280:87 , srv1759998728:87 , srv1773088374:87 , srv1918629159:87 , srv2008370346:87 , srv235120598:87 , srv252048947:87 , srv263244108:87 , srv292432957:87 , srv619633958:87 , srv691737124:87 , srv95057042:87 , srv1123604432:88 , srv1211532695:88 , srv1501337522:88 , srv1521273036:88 , srv153168181:88 , srv232501967:88 , srv391843454:88 , srv558011722:88 , srv695494231:88 , srv715310236:88 , srv785690480:88 , srv89397603:88 , srv1087664677:89 , srv126007703:89 , srv239371293:89 , srv280449050:89 , srv319521401:89 , srv538485867:89 , srv608079460:89 , srv630966934:89 , srv69952195:89 , srv1162526869:90 , srv1635990727:90 , srv2001549620:90 , srv430728535:90 , srv498012471:90 , srv816302714:90 , srv812422662:91 , srv757486050:93 , srv1927897661:513 }
2023-06-06 09:53:24,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-06-06 09:53:24,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv319521401=67, srv292432957=65, srv89397603=96, srv1717256805=40, srv1166043025=10, srv1635990727=35, srv1637402067=36, srv252048947=62, srv1363644928=17, srv430728535=72, srv95057042=99, srv217728533=55, srv812422662=91, srv263244108=63, srv1759998728=42, srv873296832=95, srv1773088374=43, srv1211532695=12, srv538485867=77, srv391843454=70, srv239923164=61, srv1405423579=21, srv695494231=84, srv69952195=85, srv1123604432=6, srv432524758=73, srv239371293=60, srv715310236=86, srv1001219384=0, srv293287682=66, srv691737124=83, srv153168181=32, srv1469921972=26, srv608079460=79, srv1501337522=29, srv235120598=59, srv1376349775=18, srv1147301365=7, srv619633958=81, srv1421918462=24, srv1675417981=39, srv76037377=88, srv1607496207=34, srv816302714=92, srv924967326=97, srv1473496604=27, srv404741158=71, srv217357422=54, srv280449050=64, srv229931174=57, srv1162526869=9, srv1421618701=23, srv498012471=76, srv1087664677=3, srv1658879280=37, srv218172699=56, srv1521273036=30, srv126007703=14, srv1584348994=33, srv1011111433=1, srv1103687522=5, srv368534157=69, srv613546467=80, srv1103063586=4, srv1918629159=48, srv1169997071=11, srv115184015=8, srv823425825=93, srv326956050=68, srv2062088207=52, srv2074984594=53, srv757486050=87, srv1397335643=19, srv2001549620=50, srv1420630196=22, srv1072728573=2, srv1219674725=13, srv1783376095=44, srv186642537=45, srv942724217=98, srv1324254384=15, srv558011722=78, srv1346214138=16, srv190361647=47, srv448967789=74, srv1757449747=41, srv763565468=89, srv1882312624=46, srv1482914116=28, srv1431708720=25, srv1927897661=49, srv857813278=94, srv489120961=75, srv232501967=58, srv785690480=90, srv1671679015=38, srv2008370346=51, srv1523554891=31, srv630966934=82, srv1402055669=20} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-06-06 09:53:24,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-06-06 09:53:24,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-06-06 09:53:24,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-06-06 09:53:24,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-06-06 09:53:24,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-06-06 09:53:24,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-06-06 09:53:24,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-06-06 09:53:24,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-06-06 09:53:24,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-06-06 09:53:24,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-06-06 09:53:24,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-06-06 09:53:24,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-06-06 09:53:24,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-06-06 09:53:24,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-06-06 09:53:24,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-06-06 09:53:24,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-06-06 09:53:24,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-06-06 09:53:24,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-06-06 09:53:24,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-06-06 09:53:24,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-06-06 09:53:24,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-06-06 09:53:24,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-06-06 09:53:24,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-06-06 09:53:24,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-06-06 09:53:24,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-06-06 09:53:24,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-06-06 09:53:24,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-06-06 09:53:24,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-06-06 09:53:24,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-06-06 09:53:24,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-06-06 09:53:24,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-06-06 09:53:24,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-06-06 09:53:24,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-06-06 09:53:24,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-06-06 09:53:24,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-06-06 09:53:24,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-06-06 09:53:24,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-06-06 09:53:24,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-06-06 09:53:24,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-06-06 09:53:24,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-06-06 09:53:24,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-06-06 09:53:24,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-06-06 09:53:24,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-06-06 09:53:24,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-06-06 09:53:24,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-06-06 09:53:24,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-06-06 09:53:24,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-06-06 09:53:24,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-06-06 09:53:24,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-06-06 09:53:24,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-06-06 09:53:24,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-06-06 09:53:24,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-06-06 09:53:24,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-06-06 09:53:24,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-06-06 09:53:24,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-06-06 09:53:24,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-06-06 09:53:24,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-06-06 09:53:24,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-06-06 09:53:24,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-06-06 09:53:24,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-06-06 09:53:24,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-06-06 09:53:24,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-06-06 09:53:24,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-06-06 09:53:24,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-06-06 09:53:24,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-06-06 09:53:24,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-06-06 09:53:24,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-06-06 09:53:24,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-06-06 09:53:24,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-06-06 09:53:24,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-06-06 09:53:24,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-06-06 09:53:24,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-06-06 09:53:24,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-06-06 09:53:24,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-06-06 09:53:24,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-06-06 09:53:24,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-06-06 09:53:24,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-06-06 09:53:24,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-06-06 09:53:24,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-06-06 09:53:24,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-06-06 09:53:24,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-06-06 09:53:24,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-06-06 09:53:24,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-06-06 09:53:24,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-06-06 09:53:24,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-06-06 09:53:24,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-06-06 09:53:24,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-06-06 09:53:24,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-06-06 09:53:24,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-06-06 09:53:24,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-06-06 09:53:24,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-06-06 09:53:24,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-06-06 09:53:24,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-06-06 09:53:24,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-06-06 09:53:24,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-06-06 09:53:24,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-06-06 09:53:24,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-06-06 09:53:24,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-06-06 09:53:24,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-06-06 09:53:24,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-06-06 09:53:24,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-06-06 09:53:24,539 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022397178843598552); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03502764660858658, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022397178843598552); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01225); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04666666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-06-06 09:53:24,539 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-06-06 09:53:24,539 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.015495223859595463, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022397178843598552); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03502764660858658, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022397178843598552); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01225); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04666666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-06-06 09:53:24,841 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 423 ms to try 5000 different iterations.  Found a solution that moves 1027 regions; Going from a computed imbalance of 0.015495223859595463 to a new imbalance of 5.350170305710608E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006789001029564207); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00999438675229417); MoveCostFunction : (multiplier=7.0, imbalance=0.11411111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006789001029564207); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0033333333333333335); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-06-06 09:53:24,842 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv430728535:80 , srv757486050:80 , srv538485867:82 , srv1103687522:83 , srv1162526869:83 , srv2001549620:83 , srv498012471:83 , srv115184015:84 , srv1166043025:84 , srv1376349775:84 , srv1421918462:84 , srv1482914116:84 , srv1607496207:84 , srv1671679015:84 , srv1759998728:84 , srv1783376095:84 , srv1918629159:84 , srv2008370346:84 , srv232501967:84 , srv292432957:84 , srv368534157:84 , srv630966934:84 , srv715310236:84 , srv1087664677:85 , srv1103063586:85 , srv1123604432:85 , srv1324254384:85 , srv1397335643:85 , srv1469921972:85 , srv1473496604:85 , srv1637402067:85 , srv1717256805:85 , srv2074984594:85 , srv217357422:85 , srv218172699:85 , srv229931174:85 , srv235120598:85 , srv280449050:85 , srv319521401:85 , srv404741158:85 , srv608079460:85 , srv613546467:85 , srv763565468:85 , srv785690480:85 , srv812422662:85 , srv1011111433:86 , srv1147301365:86 , srv1211532695:86 , srv1219674725:86 , srv126007703:86 , srv1346214138:86 , srv1363644928:86 , srv1405423579:86 , srv1420630196:86 , srv1421618701:86 , srv1431708720:86 , srv1501337522:86 , srv1521273036:86 , srv1523554891:86 , srv1584348994:86 , srv1635990727:86 , srv1658879280:86 , srv1773088374:86 , srv217728533:86 , srv252048947:86 , srv293287682:86 , srv326956050:86 , srv432524758:86 , srv448967789:86 , srv489120961:86 , srv558011722:86 , srv619633958:86 , srv691737124:86 , srv69952195:86 , srv816302714:86 , srv823425825:86 , srv873296832:86 , srv89397603:86 , srv95057042:86 , srv1001219384:87 , srv1072728573:87 , srv1169997071:87 , srv1402055669:87 , srv153168181:87 , srv1757449747:87 , srv186642537:87 , srv1882312624:87 , srv190361647:87 , srv239371293:87 , srv239923164:87 , srv263244108:87 , srv391843454:87 , srv695494231:87 , srv857813278:87 , srv924967326:87 , srv942724217:87 , srv1675417981:88 , srv2062088207:88 , srv76037377:88 , srv1927897661:545 }
2023-06-06 09:53:24,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-06-06 09:53:24,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv319521401=67, srv292432957=65, srv89397603=96, srv1717256805=40, srv1166043025=10, srv1635990727=35, srv1637402067=36, srv252048947=62, srv1363644928=17, srv430728535=72, srv95057042=99, srv217728533=55, srv812422662=91, srv263244108=63, srv1759998728=42, srv873296832=95, srv1773088374=43, srv1211532695=12, srv538485867=77, srv391843454=70, srv239923164=61, srv1405423579=21, srv695494231=84, srv69952195=85, srv1123604432=6, srv432524758=73, srv239371293=60, srv715310236=86, srv1001219384=0, srv293287682=66, srv691737124=83, srv153168181=32, srv1469921972=26, srv608079460=79, srv1501337522=29, srv235120598=59, srv1376349775=18, srv1147301365=7, srv619633958=81, srv1421918462=24, srv1675417981=39, srv76037377=88, srv1607496207=34, srv816302714=92, srv924967326=97, srv1473496604=27, srv404741158=71, srv217357422=54, srv280449050=64, srv229931174=57, srv1162526869=9, srv1421618701=23, srv498012471=76, srv1087664677=3, srv1658879280=37, srv218172699=56, srv1521273036=30, srv126007703=14, srv1584348994=33, srv1011111433=1, srv1103687522=5, srv368534157=69, srv613546467=80, srv1103063586=4, srv1918629159=48, srv1169997071=11, srv115184015=8, srv823425825=93, srv326956050=68, srv2062088207=52, srv2074984594=53, srv757486050=87, srv1397335643=19, srv2001549620=50, srv1420630196=22, srv1072728573=2, srv1219674725=13, srv1783376095=44, srv186642537=45, srv942724217=98, srv1324254384=15, srv558011722=78, srv1346214138=16, srv190361647=47, srv448967789=74, srv1757449747=41, srv763565468=89, srv1882312624=46, srv1482914116=28, srv1431708720=25, srv1927897661=49, srv857813278=94, srv489120961=75, srv232501967=58, srv785690480=90, srv1671679015=38, srv2008370346=51, srv1523554891=31, srv630966934=82, srv1402055669=20} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-06-06 09:53:24,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-06-06 09:53:24,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-06-06 09:53:24,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-06-06 09:53:24,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-06-06 09:53:24,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-06-06 09:53:24,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-06-06 09:53:24,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-06-06 09:53:24,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-06-06 09:53:24,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-06-06 09:53:24,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-06-06 09:53:24,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-06-06 09:53:24,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-06-06 09:53:24,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-06-06 09:53:24,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-06-06 09:53:24,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-06-06 09:53:24,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-06-06 09:53:24,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-06-06 09:53:24,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-06-06 09:53:24,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-06-06 09:53:24,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-06-06 09:53:24,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-06-06 09:53:24,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-06-06 09:53:24,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-06-06 09:53:24,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-06-06 09:53:24,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-06-06 09:53:24,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-06-06 09:53:24,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-06-06 09:53:24,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-06-06 09:53:24,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-06-06 09:53:24,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-06-06 09:53:24,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-06-06 09:53:24,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-06-06 09:53:24,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-06-06 09:53:24,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-06-06 09:53:24,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-06-06 09:53:24,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-06-06 09:53:24,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-06-06 09:53:24,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-06-06 09:53:24,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-06-06 09:53:24,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-06-06 09:53:24,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-06-06 09:53:24,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-06-06 09:53:24,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-06-06 09:53:24,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-06-06 09:53:24,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-06-06 09:53:24,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-06-06 09:53:24,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-06-06 09:53:24,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-06-06 09:53:24,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-06-06 09:53:24,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-06-06 09:53:24,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-06-06 09:53:24,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-06-06 09:53:24,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-06-06 09:53:24,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-06-06 09:53:24,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-06-06 09:53:24,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-06-06 09:53:24,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-06-06 09:53:24,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-06-06 09:53:24,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-06-06 09:53:24,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-06-06 09:53:24,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-06-06 09:53:24,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-06-06 09:53:24,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-06-06 09:53:24,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-06-06 09:53:24,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-06-06 09:53:24,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-06-06 09:53:24,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-06-06 09:53:24,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-06-06 09:53:24,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-06-06 09:53:24,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-06-06 09:53:24,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-06-06 09:53:24,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-06-06 09:53:24,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-06-06 09:53:24,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-06-06 09:53:24,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-06-06 09:53:24,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-06-06 09:53:24,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-06-06 09:53:24,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-06-06 09:53:24,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-06-06 09:53:24,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-06-06 09:53:24,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-06-06 09:53:24,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-06-06 09:53:24,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-06-06 09:53:24,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-06-06 09:53:24,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-06-06 09:53:24,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-06-06 09:53:24,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-06-06 09:53:24,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-06-06 09:53:24,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-06-06 09:53:24,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-06-06 09:53:24,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-06-06 09:53:24,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-06-06 09:53:24,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-06-06 09:53:24,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-06-06 09:53:24,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-06-06 09:53:24,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-06-06 09:53:24,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-06-06 09:53:24,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-06-06 09:53:24,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-06-06 09:53:24,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-06-06 09:53:24,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-06-06 09:53:25,009 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006789001029564207); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00999438675229417); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006789001029564207); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0033333333333333335); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-06-06 09:53:25,009 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-06-06 09:53:25,009 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=5.278258228702966E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006789001029564207); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00999438675229417); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006789001029564207); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0033333333333333335); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-06-06 09:53:25,389 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 510 ms to try 5000 different iterations.  Found a solution that moves 269 regions; Going from a computed imbalance of 5.278258228702966E-4 to a new imbalance of 1.9517012999820312E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017727200782610348); MoveCostFunction : (multiplier=7.0, imbalance=0.02988888888888889, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-06-06 09:53:25,390 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv2062088207:82 , srv1635990727:83 , srv280449050:83 , srv695494231:83 , srv816302714:83 , srv89397603:83 , srv1001219384:84 , srv1011111433:84 , srv1103063586:84 , srv1162526869:84 , srv1166043025:84 , srv1169997071:84 , srv1211532695:84 , srv1219674725:84 , srv126007703:84 , srv1346214138:84 , srv1376349775:84 , srv1421618701:84 , srv1469921972:84 , srv153168181:84 , srv1717256805:84 , srv1757449747:84 , srv1918629159:84 , srv217728533:84 , srv239923164:84 , srv252048947:84 , srv263244108:84 , srv326956050:84 , srv391843454:84 , srv430728535:84 , srv432524758:84 , srv498012471:84 , srv538485867:84 , srv613546467:84 , srv691737124:84 , srv757486050:84 , srv76037377:84 , srv785690480:84 , srv812422662:84 , srv873296832:84 , srv924967326:84 , srv95057042:84 , srv1072728573:85 , srv1087664677:85 , srv1103687522:85 , srv1123604432:85 , srv1147301365:85 , srv1324254384:85 , srv1363644928:85 , srv1402055669:85 , srv1405423579:85 , srv1473496604:85 , srv1482914116:85 , srv1501337522:85 , srv1521273036:85 , srv1523554891:85 , srv1637402067:85 , srv1658879280:85 , srv1671679015:85 , srv1675417981:85 , srv1759998728:85 , srv1773088374:85 , srv1783376095:85 , srv186642537:85 , srv190361647:85 , srv2001549620:85 , srv2008370346:85 , srv2074984594:85 , srv217357422:85 , srv218172699:85 , srv229931174:85 , srv232501967:85 , srv239371293:85 , srv292432957:85 , srv319521401:85 , srv368534157:85 , srv404741158:85 , srv448967789:85 , srv489120961:85 , srv558011722:85 , srv608079460:85 , srv619633958:85 , srv630966934:85 , srv69952195:85 , srv715310236:85 , srv763565468:85 , srv823425825:85 , srv857813278:85 , srv942724217:85 , srv115184015:86 , srv1397335643:86 , srv1420630196:86 , srv1421918462:86 , srv1431708720:86 , srv1584348994:86 , srv1607496207:86 , srv1882312624:86 , srv235120598:86 , srv293287682:86 , srv1927897661:624 }
2023-06-06 09:53:25,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-06-06 09:53:25,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv319521401=67, srv292432957=65, srv89397603=96, srv1717256805=40, srv1166043025=10, srv1635990727=35, srv1637402067=36, srv252048947=62, srv1363644928=17, srv430728535=72, srv95057042=99, srv217728533=55, srv812422662=91, srv263244108=63, srv1759998728=42, srv873296832=95, srv1773088374=43, srv1211532695=12, srv538485867=77, srv391843454=70, srv239923164=61, srv1405423579=21, srv695494231=84, srv69952195=85, srv1123604432=6, srv432524758=73, srv239371293=60, srv715310236=86, srv1001219384=0, srv293287682=66, srv691737124=83, srv153168181=32, srv1469921972=26, srv608079460=79, srv1501337522=29, srv235120598=59, srv1376349775=18, srv1147301365=7, srv619633958=81, srv1421918462=24, srv1675417981=39, srv76037377=88, srv1607496207=34, srv816302714=92, srv924967326=97, srv1473496604=27, srv404741158=71, srv217357422=54, srv280449050=64, srv229931174=57, srv1162526869=9, srv1421618701=23, srv498012471=76, srv1087664677=3, srv1658879280=37, srv218172699=56, srv1521273036=30, srv126007703=14, srv1584348994=33, srv1011111433=1, srv1103687522=5, srv368534157=69, srv613546467=80, srv1103063586=4, srv1918629159=48, srv1169997071=11, srv115184015=8, srv823425825=93, srv326956050=68, srv2062088207=52, srv2074984594=53, srv757486050=87, srv1397335643=19, srv2001549620=50, srv1420630196=22, srv1072728573=2, srv1219674725=13, srv1783376095=44, srv186642537=45, srv942724217=98, srv1324254384=15, srv558011722=78, srv1346214138=16, srv190361647=47, srv448967789=74, srv1757449747=41, srv763565468=89, srv1882312624=46, srv1482914116=28, srv1431708720=25, srv1927897661=49, srv857813278=94, srv489120961=75, srv232501967=58, srv785690480=90, srv1671679015=38, srv2008370346=51, srv1523554891=31, srv630966934=82, srv1402055669=20} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-06-06 09:53:25,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-06-06 09:53:25,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-06-06 09:53:25,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-06-06 09:53:25,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-06-06 09:53:25,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-06-06 09:53:25,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-06-06 09:53:25,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-06-06 09:53:25,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-06-06 09:53:25,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-06-06 09:53:25,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-06-06 09:53:25,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-06-06 09:53:25,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-06-06 09:53:25,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-06-06 09:53:25,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-06-06 09:53:25,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-06-06 09:53:25,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-06-06 09:53:25,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-06-06 09:53:25,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-06-06 09:53:25,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-06-06 09:53:25,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-06-06 09:53:25,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-06-06 09:53:25,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-06-06 09:53:25,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-06-06 09:53:25,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-06-06 09:53:25,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-06-06 09:53:25,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-06-06 09:53:25,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-06-06 09:53:25,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-06-06 09:53:25,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-06-06 09:53:25,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-06-06 09:53:25,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-06-06 09:53:25,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-06-06 09:53:25,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-06-06 09:53:25,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-06-06 09:53:25,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-06-06 09:53:25,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-06-06 09:53:25,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-06-06 09:53:25,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-06-06 09:53:25,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-06-06 09:53:25,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-06-06 09:53:25,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-06-06 09:53:25,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-06-06 09:53:25,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-06-06 09:53:25,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-06-06 09:53:25,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-06-06 09:53:25,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-06-06 09:53:25,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-06-06 09:53:25,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-06-06 09:53:25,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-06-06 09:53:25,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-06-06 09:53:25,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-06-06 09:53:25,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-06-06 09:53:25,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-06-06 09:53:25,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-06-06 09:53:25,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-06-06 09:53:25,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-06-06 09:53:25,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-06-06 09:53:25,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-06-06 09:53:25,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-06-06 09:53:25,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-06-06 09:53:25,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-06-06 09:53:25,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-06-06 09:53:25,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-06-06 09:53:25,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-06-06 09:53:25,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-06-06 09:53:25,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-06-06 09:53:25,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-06-06 09:53:25,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-06-06 09:53:25,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-06-06 09:53:25,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-06-06 09:53:25,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-06-06 09:53:25,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-06-06 09:53:25,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-06-06 09:53:25,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-06-06 09:53:25,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-06-06 09:53:25,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-06-06 09:53:25,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-06-06 09:53:25,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-06-06 09:53:25,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-06-06 09:53:25,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-06-06 09:53:25,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-06-06 09:53:25,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-06-06 09:53:25,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-06-06 09:53:25,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-06-06 09:53:25,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-06-06 09:53:25,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-06-06 09:53:25,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-06-06 09:53:25,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-06-06 09:53:25,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-06-06 09:53:25,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-06-06 09:53:25,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-06-06 09:53:25,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-06-06 09:53:25,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-06-06 09:53:25,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-06-06 09:53:25,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-06-06 09:53:25,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-06-06 09:53:25,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-06-06 09:53:25,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-06-06 09:53:25,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-06-06 09:53:25,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-06-06 09:53:25,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-06-06 09:53:25,518 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.7633434741294947E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017727200782610348); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-06-06 09:53:25,518 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv2062088207:82 , srv1635990727:83 , srv280449050:83 , srv695494231:83 , srv816302714:83 , srv89397603:83 , srv1001219384:84 , srv1011111433:84 , srv1103063586:84 , srv1162526869:84 , srv1166043025:84 , srv1169997071:84 , srv1211532695:84 , srv1219674725:84 , srv126007703:84 , srv1346214138:84 , srv1376349775:84 , srv1421618701:84 , srv1469921972:84 , srv153168181:84 , srv1717256805:84 , srv1757449747:84 , srv1918629159:84 , srv217728533:84 , srv239923164:84 , srv252048947:84 , srv263244108:84 , srv326956050:84 , srv391843454:84 , srv430728535:84 , srv432524758:84 , srv498012471:84 , srv538485867:84 , srv613546467:84 , srv691737124:84 , srv757486050:84 , srv76037377:84 , srv785690480:84 , srv812422662:84 , srv873296832:84 , srv924967326:84 , srv95057042:84 , srv1072728573:85 , srv1087664677:85 , srv1103687522:85 , srv1123604432:85 , srv1147301365:85 , srv1324254384:85 , srv1363644928:85 , srv1402055669:85 , srv1405423579:85 , srv1473496604:85 , srv1482914116:85 , srv1501337522:85 , srv1521273036:85 , srv1523554891:85 , srv1637402067:85 , srv1658879280:85 , srv1671679015:85 , srv1675417981:85 , srv1759998728:85 , srv1773088374:85 , srv1783376095:85 , srv186642537:85 , srv190361647:85 , srv2001549620:85 , srv2008370346:85 , srv2074984594:85 , srv217357422:85 , srv218172699:85 , srv229931174:85 , srv232501967:85 , srv239371293:85 , srv292432957:85 , srv319521401:85 , srv368534157:85 , srv404741158:85 , srv448967789:85 , srv489120961:85 , srv558011722:85 , srv608079460:85 , srv619633958:85 , srv630966934:85 , srv69952195:85 , srv715310236:85 , srv763565468:85 , srv823425825:85 , srv857813278:85 , srv942724217:85 , srv115184015:86 , srv1397335643:86 , srv1420630196:86 , srv1421918462:86 , srv1431708720:86 , srv1584348994:86 , srv1607496207:86 , srv1882312624:86 , srv235120598:86 , srv293287682:86 , srv1927897661:624 }
2023-06-06 09:53:25,548 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=346 (was 280) - SystemLoadAverage LEAK? -, ProcessCount=176 (was 176), AvailableMemoryMB=7485 (was 7981)
2023-06-06 09:53:25,558 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=346, ProcessCount=176, AvailableMemoryMB=7484
2023-06-06 09:53:25,558 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-06-06 09:53:25,559 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-06-06 09:53:25,567 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1001199907:84 , srv1041497901:84 , srv110125837:84 , srv1103337557:84 , srv1105311675:84 , srv115959349:84 , srv1260132507:84 , srv1289071124:84 , srv1444990619:84 , srv1492944014:84 , srv1570821490:84 , srv1669737548:84 , srv1851319885:84 , srv1861212111:84 , srv1884110440:84 , srv1895523585:84 , srv1907590583:84 , srv20559662:84 , srv217937520:84 , srv262651670:84 , srv353295965:84 , srv508789750:84 , srv536109991:84 , srv622644682:84 , srv648049429:84 , srv748799050:84 , srv768161177:84 , srv875147783:84 , srv98640937:84 , srv1060787494:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-06-06 09:53:25,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-06-06 09:53:25,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1105311675=5, srv1260132507=7, srv1492944014=10, srv1895523585=16, srv508789750=22, srv1001199907=0, srv1907590583=17, srv353295965=21, srv1060787494=2, srv98640937=29, srv1851319885=13, srv1289071124=8, srv115959349=6, srv748799050=26, srv648049429=25, srv1861212111=14, srv622644682=24, srv1570821490=11, srv536109991=23, srv875147783=28, srv217937520=19, srv1884110440=15, srv262651670=20, srv110125837=3, srv1103337557=4, srv20559662=18, srv1041497901=1, srv1669737548=12, srv768161177=27, srv1444990619=9} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-06-06 09:53:25,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-06-06 09:53:25,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-06-06 09:53:25,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-06-06 09:53:25,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-06-06 09:53:25,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-06-06 09:53:25,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-06-06 09:53:25,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-06-06 09:53:25,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-06-06 09:53:25,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-06-06 09:53:25,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-06-06 09:53:25,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-06-06 09:53:25,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-06-06 09:53:25,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-06-06 09:53:25,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-06-06 09:53:25,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-06-06 09:53:25,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-06-06 09:53:25,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-06-06 09:53:25,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-06-06 09:53:25,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-06-06 09:53:25,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-06-06 09:53:25,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-06-06 09:53:25,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-06-06 09:53:25,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-06-06 09:53:25,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-06-06 09:53:25,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-06-06 09:53:25,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-06-06 09:53:25,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-06-06 09:53:25,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-06-06 09:53:25,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-06-06 09:53:25,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-06-06 09:53:25,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-06-06 09:53:25,596 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1225208116402865, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-06-06 09:53:25,597 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912579102341085, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1225208116402865, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-06-06 09:55:25,583 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5551736 different iterations.  Found a solution that moves 1870 regions; Going from a computed imbalance of 0.9912579102341085 to a new imbalance of 4.3646732880327594E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6925925925925925, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-06-06 09:55:25,589 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1001199907:90 , srv1041497901:90 , srv1060787494:90 , srv110125837:90 , srv1103337557:90 , srv1105311675:90 , srv115959349:90 , srv1260132507:90 , srv1289071124:90 , srv1444990619:90 , srv1492944014:90 , srv1570821490:90 , srv1669737548:90 , srv1851319885:90 , srv1861212111:90 , srv1884110440:90 , srv1895523585:90 , srv1907590583:90 , srv20559662:90 , srv217937520:90 , srv262651670:90 , srv353295965:90 , srv508789750:90 , srv536109991:90 , srv622644682:90 , srv648049429:90 , srv748799050:90 , srv768161177:90 , srv875147783:90 , srv98640937:90 }
2023-06-06 09:55:25,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-06-06 09:55:25,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1105311675=5, srv1260132507=7, srv1492944014=10, srv1895523585=16, srv508789750=22, srv1001199907=0, srv1907590583=17, srv353295965=21, srv1060787494=2, srv98640937=29, srv1851319885=13, srv1289071124=8, srv115959349=6, srv748799050=26, srv648049429=25, srv1861212111=14, srv622644682=24, srv1570821490=11, srv536109991=23, srv875147783=28, srv217937520=19, srv1884110440=15, srv262651670=20, srv110125837=3, srv1103337557=4, srv20559662=18, srv1041497901=1, srv1669737548=12, srv768161177=27, srv1444990619=9} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-06-06 09:55:25,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-06-06 09:55:25,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-06-06 09:55:25,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-06-06 09:55:25,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-06-06 09:55:25,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-06-06 09:55:25,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-06-06 09:55:25,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-06-06 09:55:25,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-06-06 09:55:25,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-06-06 09:55:25,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-06-06 09:55:25,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-06-06 09:55:25,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-06-06 09:55:25,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-06-06 09:55:25,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-06-06 09:55:25,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-06-06 09:55:25,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-06-06 09:55:25,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-06-06 09:55:25,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-06-06 09:55:25,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-06-06 09:55:25,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-06-06 09:55:25,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-06-06 09:55:25,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-06-06 09:55:25,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-06-06 09:55:25,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-06-06 09:55:25,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-06-06 09:55:25,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-06-06 09:55:25,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-06-06 09:55:25,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-06-06 09:55:25,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-06-06 09:55:25,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-06-06 09:55:25,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-06-06 09:55:25,606 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-06-06 09:55:25,606 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1001199907:90 , srv1041497901:90 , srv1060787494:90 , srv110125837:90 , srv1103337557:90 , srv1105311675:90 , srv115959349:90 , srv1260132507:90 , srv1289071124:90 , srv1444990619:90 , srv1492944014:90 , srv1570821490:90 , srv1669737548:90 , srv1851319885:90 , srv1861212111:90 , srv1884110440:90 , srv1895523585:90 , srv1907590583:90 , srv20559662:90 , srv217937520:90 , srv262651670:90 , srv353295965:90 , srv508789750:90 , srv536109991:90 , srv622644682:90 , srv648049429:90 , srv748799050:90 , srv768161177:90 , srv875147783:90 , srv98640937:90 }
2023-06-06 09:55:25,619 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=190 (was 346), ProcessCount=173 (was 176), AvailableMemoryMB=6765 (was 7484)