2023-05-16 09:53:24,725 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-05-16 09:53:25,052 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-05-16 09:53:25,169 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-05-16 09:53:25,238 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-16 09:53:25,248 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-05-16 09:53:25,264 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=264, MaxFileDescriptor=60000, SystemLoadAverage=350, ProcessCount=172, AvailableMemoryMB=13964
2023-05-16 09:53:25,267 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-16 09:53:25,267 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-05-16 09:53:25,437 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1004953103:84 , srv1006595598:84 , srv1050608744:84 , srv1067463010:84 , srv1137114292:84 , srv1140364989:84 , srv1153045974:84 , srv1178174874:84 , srv1183423838:84 , srv1210395838:84 , srv1281806801:84 , srv1316561937:84 , srv1339147962:84 , srv1364061727:84 , srv1377958486:84 , srv1385540086:84 , srv1399646197:84 , srv1405523351:84 , srv1415722315:84 , srv1447994151:84 , srv1487260858:84 , srv1500683094:84 , srv1502798044:84 , srv1507788191:84 , srv1519375836:84 , srv1522825815:84 , srv1549257265:84 , srv1581004476:84 , srv161438503:84 , srv1657040172:84 , srv1677549317:84 , srv1684531382:84 , srv172732821:84 , srv1734357949:84 , srv1737195973:84 , srv17427872:84 , srv1777579649:84 , srv1795134188:84 , srv1808550494:84 , srv1872073586:84 , srv1880172318:84 , srv1909170097:84 , srv191998867:84 , srv1924644169:84 , srv2015208612:84 , srv2024031992:84 , srv2032011782:84 , srv2036466530:84 , srv2046627257:84 , srv2051106511:84 , srv2065733404:84 , srv2116685327:84 , srv2136825552:84 , srv254264596:84 , srv277425780:84 , srv293934176:84 , srv304683096:84 , srv329168658:84 , srv329901127:84 , srv335305788:84 , srv359133873:84 , srv375306944:84 , srv399935515:84 , srv407789080:84 , srv421881687:84 , srv44171577:84 , srv444517227:84 , srv450541360:84 , srv46595306:84 , srv498831413:84 , srv509323606:84 , srv511398101:84 , srv516515022:84 , srv527198136:84 , srv529079762:84 , srv531311072:84 , srv595837131:84 , srv671283221:84 , srv689772699:84 , srv695595066:84 , srv712477873:84 , srv768278717:84 , srv775035430:84 , srv83908066:84 , srv842367805:84 , srv865297472:84 , srv892807162:84 , srv901565069:84 , srv908342079:84 , srv910495922:84 , srv91965146:84 , srv922632106:84 , srv927748624:84 , srv930192434:84 , srv931864120:84 , srv940020697:84 , srv954018157:84 , srv963089777:84 , srv995316746:84 , srv1643441367:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-05-16 09:53:25,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-16 09:53:25,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1507788191=23, srv1500683094=21, srv1677549317=31, srv768278717=82, srv1909170097=42, srv527198136=74, srv695595066=80, srv910495922=90, srv2015208612=45, srv1067463010=3, srv1153045974=6, srv83908066=84, srv712477873=81, srv1183423838=8, srv172732821=33, srv940020697=96, srv375306944=62, srv2046627257=49, srv421881687=65, srv775035430=83, srv931864120=95, srv1519375836=24, srv1415722315=18, srv1880172318=41, srv46595306=69, srv1737195973=35, srv927748624=93, srv1405523351=17, srv329901127=59, srv671283221=78, srv511398101=72, srv91965146=91, srv1549257265=26, srv1643441367=29, srv1808550494=39, srv1316561937=11, srv2051106511=50, srv892807162=87, srv1924644169=44, srv191998867=43, srv450541360=68, srv359133873=61, srv1581004476=27, srv1522825815=25, srv304683096=57, srv1140364989=5, srv2136825552=53, srv1487260858=20, srv1377958486=14, srv595837131=77, srv1006595598=1, srv407789080=64, srv2065733404=51, srv954018157=97, srv930192434=94, srv963089777=98, srv335305788=60, srv1734357949=34, srv44171577=66, srv161438503=28, srv1447994151=19, srv922632106=92, srv2032011782=47, srv1281806801=10, srv498831413=70, srv1872073586=40, srv399935515=63, srv516515022=73, srv901565069=88, srv1684531382=32, srv1795134188=38, srv1210395838=9, srv17427872=36, srv1137114292=4, srv444517227=67, srv254264596=54, srv2036466530=48, srv995316746=99, srv1399646197=16, srv1339147962=12, srv1364061727=13, srv689772699=79, srv1657040172=30, srv1777579649=37, srv1385540086=15, srv2116685327=52, srv1502798044=22, srv1050608744=2, srv277425780=55, srv1004953103=0, srv293934176=56, srv329168658=58, srv2024031992=46, srv908342079=89, srv509323606=71, srv842367805=85, srv865297472=86, srv531311072=76, srv529079762=75, srv1178174874=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-16 09:53:25,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-16 09:53:25,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-16 09:53:25,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-16 09:53:25,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-16 09:53:25,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-16 09:53:25,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-16 09:53:25,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-16 09:53:25,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-16 09:53:25,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-16 09:53:25,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-16 09:53:25,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-16 09:53:25,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-16 09:53:25,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-16 09:53:25,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-16 09:53:25,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-16 09:53:25,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-16 09:53:25,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-16 09:53:25,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-16 09:53:25,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-16 09:53:25,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-16 09:53:25,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-16 09:53:25,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-16 09:53:25,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-16 09:53:25,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-16 09:53:25,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-16 09:53:25,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-16 09:53:25,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-16 09:53:25,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-16 09:53:25,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-16 09:53:25,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-16 09:53:25,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-16 09:53:25,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-16 09:53:25,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-16 09:53:25,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-16 09:53:25,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-16 09:53:25,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-16 09:53:25,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-16 09:53:25,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-16 09:53:25,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-16 09:53:25,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-16 09:53:25,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-16 09:53:25,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-16 09:53:25,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-16 09:53:25,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-16 09:53:25,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-16 09:53:25,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-16 09:53:25,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-16 09:53:25,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-16 09:53:25,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-16 09:53:25,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-16 09:53:25,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-16 09:53:25,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-16 09:53:25,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-16 09:53:25,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-16 09:53:25,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-16 09:53:25,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-16 09:53:25,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-16 09:53:25,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-16 09:53:25,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-16 09:53:25,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-16 09:53:25,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-16 09:53:25,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-16 09:53:25,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-16 09:53:25,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-16 09:53:25,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-16 09:53:25,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-16 09:53:25,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-16 09:53:25,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-16 09:53:25,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-16 09:53:25,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-16 09:53:25,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-16 09:53:25,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-16 09:53:25,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-16 09:53:25,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-16 09:53:25,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-16 09:53:25,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-16 09:53:25,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-16 09:53:25,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-16 09:53:25,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-16 09:53:25,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-16 09:53:25,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-16 09:53:25,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-16 09:53:25,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-16 09:53:25,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-16 09:53:25,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-16 09:53:25,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-16 09:53:25,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-16 09:53:25,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-16 09:53:25,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-16 09:53:25,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-16 09:53:25,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-16 09:53:25,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-16 09:53:25,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-16 09:53:25,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-16 09:53:25,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-16 09:53:25,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-16 09:53:25,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-16 09:53:25,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-16 09:53:25,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-16 09:53:25,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-16 09:53:25,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-16 09:53:25,788 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-16 09:53:25,788 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-16 09:53:25,788 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-16 09:53:26,592 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 1031 ms to try 5000 different iterations.  Found a solution that moves 4678 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1785390033372611. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03879273024280109, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0577019704705486, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5197777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03879273024280109, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17366666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24116666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-16 09:53:26,626 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1522825815:80 , srv2051106511:80 , srv1405523351:81 , srv1677549317:82 , srv1734357949:82 , srv931864120:82 , srv1137114292:83 , srv1281806801:83 , srv1385540086:83 , srv1399646197:83 , srv1487260858:83 , srv1737195973:83 , srv1924644169:83 , srv2015208612:83 , srv329168658:83 , srv689772699:83 , srv768278717:83 , srv1684531382:84 , srv1777579649:84 , srv2036466530:84 , srv407789080:84 , srv83908066:84 , srv1377958486:85 , srv1502798044:85 , srv17427872:85 , srv1795134188:85 , srv1808550494:85 , srv1872073586:85 , srv2024031992:85 , srv254264596:85 , srv329901127:85 , srv335305788:85 , srv421881687:85 , srv775035430:85 , srv842367805:85 , srv927748624:85 , srv995316746:85 , srv1067463010:86 , srv1153045974:86 , srv1178174874:86 , srv1183423838:86 , srv1364061727:86 , srv1507788191:86 , srv1519375836:86 , srv1549257265:86 , srv161438503:86 , srv1657040172:86 , srv2116685327:86 , srv293934176:86 , srv375306944:86 , srv44171577:86 , srv511398101:86 , srv531311072:86 , srv910495922:86 , srv91965146:86 , srv922632106:86 , srv930192434:86 , srv940020697:86 , srv1004953103:87 , srv1581004476:87 , srv172732821:87 , srv191998867:87 , srv399935515:87 , srv450541360:87 , srv509323606:87 , srv527198136:87 , srv892807162:87 , srv1006595598:88 , srv1050608744:88 , srv1316561937:88 , srv1415722315:88 , srv2032011782:88 , srv2046627257:88 , srv2136825552:88 , srv277425780:88 , srv529079762:88 , srv595837131:88 , srv901565069:88 , srv908342079:88 , srv304683096:89 , srv444517227:89 , srv516515022:89 , srv712477873:89 , srv865297472:89 , srv1210395838:90 , srv1339147962:90 , srv1880172318:90 , srv1909170097:90 , srv359133873:90 , srv498831413:90 , srv671283221:90 , srv695595066:90 , srv954018157:90 , srv1447994151:91 , srv2065733404:91 , srv46595306:92 , srv963089777:93 , srv1140364989:102 , srv1500683094:102 , srv1643441367:434 }
2023-05-16 09:53:26,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-16 09:53:26,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1507788191=23, srv1500683094=21, srv1677549317=31, srv768278717=82, srv1909170097=42, srv527198136=74, srv695595066=80, srv910495922=90, srv2015208612=45, srv1067463010=3, srv1153045974=6, srv83908066=84, srv712477873=81, srv1183423838=8, srv172732821=33, srv940020697=96, srv375306944=62, srv2046627257=49, srv421881687=65, srv775035430=83, srv931864120=95, srv1519375836=24, srv1415722315=18, srv1880172318=41, srv46595306=69, srv1737195973=35, srv927748624=93, srv1405523351=17, srv329901127=59, srv671283221=78, srv511398101=72, srv91965146=91, srv1549257265=26, srv1643441367=29, srv1808550494=39, srv1316561937=11, srv2051106511=50, srv892807162=87, srv1924644169=44, srv191998867=43, srv450541360=68, srv359133873=61, srv1581004476=27, srv1522825815=25, srv304683096=57, srv1140364989=5, srv2136825552=53, srv1487260858=20, srv1377958486=14, srv595837131=77, srv1006595598=1, srv407789080=64, srv2065733404=51, srv954018157=97, srv930192434=94, srv963089777=98, srv335305788=60, srv1734357949=34, srv44171577=66, srv161438503=28, srv1447994151=19, srv922632106=92, srv2032011782=47, srv1281806801=10, srv498831413=70, srv1872073586=40, srv399935515=63, srv516515022=73, srv901565069=88, srv1684531382=32, srv1795134188=38, srv1210395838=9, srv17427872=36, srv1137114292=4, srv444517227=67, srv254264596=54, srv2036466530=48, srv995316746=99, srv1399646197=16, srv1339147962=12, srv1364061727=13, srv689772699=79, srv1657040172=30, srv1777579649=37, srv1385540086=15, srv2116685327=52, srv1502798044=22, srv1050608744=2, srv277425780=55, srv1004953103=0, srv293934176=56, srv329168658=58, srv2024031992=46, srv908342079=89, srv509323606=71, srv842367805=85, srv865297472=86, srv531311072=76, srv529079762=75, srv1178174874=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-16 09:53:26,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-16 09:53:26,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-16 09:53:26,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-16 09:53:26,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-16 09:53:26,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-16 09:53:26,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-16 09:53:26,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-16 09:53:26,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-16 09:53:26,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-16 09:53:26,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-16 09:53:26,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-16 09:53:26,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-16 09:53:26,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-16 09:53:26,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-16 09:53:26,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-16 09:53:26,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-16 09:53:26,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-16 09:53:26,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-16 09:53:26,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-16 09:53:26,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-16 09:53:26,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-16 09:53:26,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-16 09:53:26,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-16 09:53:26,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-16 09:53:26,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-16 09:53:26,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-16 09:53:26,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-16 09:53:26,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-16 09:53:26,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-16 09:53:26,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-16 09:53:26,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-16 09:53:26,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-16 09:53:26,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-16 09:53:26,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-16 09:53:26,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-16 09:53:26,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-16 09:53:26,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-16 09:53:26,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-16 09:53:26,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-16 09:53:26,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-16 09:53:26,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-16 09:53:26,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-16 09:53:26,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-16 09:53:26,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-16 09:53:26,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-16 09:53:26,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-16 09:53:26,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-16 09:53:26,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-16 09:53:26,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-16 09:53:26,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-16 09:53:26,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-16 09:53:26,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-16 09:53:26,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-16 09:53:26,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-16 09:53:26,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-16 09:53:26,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-16 09:53:26,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-16 09:53:26,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-16 09:53:26,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-16 09:53:26,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-16 09:53:26,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-16 09:53:26,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-16 09:53:26,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-16 09:53:26,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-16 09:53:26,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-16 09:53:26,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-16 09:53:26,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-16 09:53:26,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-16 09:53:26,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-16 09:53:26,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-16 09:53:26,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-16 09:53:26,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-16 09:53:26,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-16 09:53:26,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-16 09:53:26,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-16 09:53:26,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-16 09:53:26,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-16 09:53:26,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-16 09:53:26,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-16 09:53:26,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-16 09:53:26,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-16 09:53:26,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-16 09:53:26,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-16 09:53:26,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-16 09:53:26,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-16 09:53:26,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-16 09:53:26,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-16 09:53:26,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-16 09:53:26,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-16 09:53:26,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-16 09:53:26,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-16 09:53:26,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-16 09:53:26,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-16 09:53:26,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-16 09:53:26,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-16 09:53:26,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-16 09:53:26,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-16 09:53:26,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-16 09:53:26,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-16 09:53:26,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-16 09:53:26,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-16 09:53:26,920 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03879273024280109, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0577019704705486, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03879273024280109, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17366666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24116666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-16 09:53:26,920 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-16 09:53:26,920 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.1785062472811519, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03879273024280109, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0577019704705486, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03879273024280109, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17366666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24116666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-16 09:53:27,281 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 565 ms to try 5000 different iterations.  Found a solution that moves 2664 regions; Going from a computed imbalance of 0.1785062472811519 to a new imbalance of 0.01610030767501809. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021850425108932717); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.034223796369410926, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.296, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021850425108932717); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01325); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04325, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-16 09:53:27,292 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1140364989:68 , srv1500683094:71 , srv46595306:78 , srv695595066:80 , srv1006595598:81 , srv359133873:81 , srv963089777:81 , srv1210395838:82 , srv1339147962:82 , srv1415722315:82 , srv1909170097:82 , srv671283221:82 , srv865297472:82 , srv954018157:82 , srv1050608744:83 , srv1316561937:83 , srv1581004476:83 , srv1880172318:83 , srv2046627257:83 , srv2065733404:83 , srv595837131:83 , srv712477873:83 , srv1507788191:84 , srv191998867:84 , srv2136825552:84 , srv399935515:84 , srv444517227:84 , srv91965146:84 , srv922632106:84 , srv930192434:84 , srv1004953103:85 , srv1067463010:85 , srv17427872:85 , srv1872073586:85 , srv2032011782:85 , srv304683096:85 , srv375306944:85 , srv44171577:85 , srv498831413:85 , srv516515022:85 , srv527198136:85 , srv529079762:85 , srv531311072:85 , srv908342079:85 , srv940020697:85 , srv1137114292:86 , srv1183423838:86 , srv1549257265:86 , srv161438503:86 , srv1808550494:86 , srv277425780:86 , srv329901127:86 , srv775035430:86 , srv83908066:86 , srv901565069:86 , srv995316746:86 , srv1178174874:87 , srv1377958486:87 , srv1502798044:87 , srv1519375836:87 , srv2024031992:87 , srv2116685327:87 , srv254264596:87 , srv293934176:87 , srv407789080:87 , srv421881687:87 , srv450541360:87 , srv509323606:87 , srv511398101:87 , srv842367805:87 , srv892807162:87 , srv1153045974:88 , srv1281806801:88 , srv1385540086:88 , srv1399646197:88 , srv1447994151:88 , srv1657040172:88 , srv1677549317:88 , srv1684531382:88 , srv2036466530:88 , srv335305788:88 , srv689772699:88 , srv768278717:88 , srv910495922:88 , srv927748624:88 , srv1487260858:89 , srv172732821:89 , srv1734357949:89 , srv1737195973:89 , srv2015208612:89 , srv329168658:89 , srv1777579649:90 , srv1795134188:90 , srv1924644169:90 , srv931864120:90 , srv1364061727:91 , srv1405523351:91 , srv1522825815:91 , srv2051106511:91 , srv1643441367:534 }
2023-05-16 09:53:27,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-16 09:53:27,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1507788191=23, srv1500683094=21, srv1677549317=31, srv768278717=82, srv1909170097=42, srv527198136=74, srv695595066=80, srv910495922=90, srv2015208612=45, srv1067463010=3, srv1153045974=6, srv83908066=84, srv712477873=81, srv1183423838=8, srv172732821=33, srv940020697=96, srv375306944=62, srv2046627257=49, srv421881687=65, srv775035430=83, srv931864120=95, srv1519375836=24, srv1415722315=18, srv1880172318=41, srv46595306=69, srv1737195973=35, srv927748624=93, srv1405523351=17, srv329901127=59, srv671283221=78, srv511398101=72, srv91965146=91, srv1549257265=26, srv1643441367=29, srv1808550494=39, srv1316561937=11, srv2051106511=50, srv892807162=87, srv1924644169=44, srv191998867=43, srv450541360=68, srv359133873=61, srv1581004476=27, srv1522825815=25, srv304683096=57, srv1140364989=5, srv2136825552=53, srv1487260858=20, srv1377958486=14, srv595837131=77, srv1006595598=1, srv407789080=64, srv2065733404=51, srv954018157=97, srv930192434=94, srv963089777=98, srv335305788=60, srv1734357949=34, srv44171577=66, srv161438503=28, srv1447994151=19, srv922632106=92, srv2032011782=47, srv1281806801=10, srv498831413=70, srv1872073586=40, srv399935515=63, srv516515022=73, srv901565069=88, srv1684531382=32, srv1795134188=38, srv1210395838=9, srv17427872=36, srv1137114292=4, srv444517227=67, srv254264596=54, srv2036466530=48, srv995316746=99, srv1399646197=16, srv1339147962=12, srv1364061727=13, srv689772699=79, srv1657040172=30, srv1777579649=37, srv1385540086=15, srv2116685327=52, srv1502798044=22, srv1050608744=2, srv277425780=55, srv1004953103=0, srv293934176=56, srv329168658=58, srv2024031992=46, srv908342079=89, srv509323606=71, srv842367805=85, srv865297472=86, srv531311072=76, srv529079762=75, srv1178174874=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-16 09:53:27,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-16 09:53:27,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-16 09:53:27,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-16 09:53:27,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-16 09:53:27,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-16 09:53:27,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-16 09:53:27,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-16 09:53:27,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-16 09:53:27,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-16 09:53:27,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-16 09:53:27,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-16 09:53:27,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-16 09:53:27,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-16 09:53:27,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-16 09:53:27,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-16 09:53:27,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-16 09:53:27,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-16 09:53:27,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-16 09:53:27,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-16 09:53:27,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-16 09:53:27,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-16 09:53:27,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-16 09:53:27,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-16 09:53:27,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-16 09:53:27,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-16 09:53:27,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-16 09:53:27,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-16 09:53:27,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-16 09:53:27,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-16 09:53:27,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-16 09:53:27,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-16 09:53:27,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-16 09:53:27,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-16 09:53:27,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-16 09:53:27,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-16 09:53:27,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-16 09:53:27,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-16 09:53:27,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-16 09:53:27,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-16 09:53:27,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-16 09:53:27,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-16 09:53:27,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-16 09:53:27,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-16 09:53:27,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-16 09:53:27,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-16 09:53:27,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-16 09:53:27,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-16 09:53:27,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-16 09:53:27,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-16 09:53:27,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-16 09:53:27,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-16 09:53:27,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-16 09:53:27,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-16 09:53:27,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-16 09:53:27,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-16 09:53:27,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-16 09:53:27,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-16 09:53:27,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-16 09:53:27,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-16 09:53:27,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-16 09:53:27,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-16 09:53:27,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-16 09:53:27,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-16 09:53:27,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-16 09:53:27,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-16 09:53:27,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-16 09:53:27,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-16 09:53:27,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-16 09:53:27,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-16 09:53:27,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-16 09:53:27,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-16 09:53:27,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-16 09:53:27,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-16 09:53:27,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-16 09:53:27,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-16 09:53:27,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-16 09:53:27,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-16 09:53:27,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-16 09:53:27,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-16 09:53:27,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-16 09:53:27,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-16 09:53:27,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-16 09:53:27,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-16 09:53:27,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-16 09:53:27,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-16 09:53:27,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-16 09:53:27,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-16 09:53:27,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-16 09:53:27,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-16 09:53:27,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-16 09:53:27,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-16 09:53:27,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-16 09:53:27,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-16 09:53:27,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-16 09:53:27,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-16 09:53:27,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-16 09:53:27,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-16 09:53:27,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-16 09:53:27,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-16 09:53:27,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-16 09:53:27,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-16 09:53:27,450 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021850425108932717); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.034223796369410926, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021850425108932717); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01325); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04325, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-16 09:53:27,451 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-16 09:53:27,451 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.016081653948323997, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021850425108932717); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.034223796369410926, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021850425108932717); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01325); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04325, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-16 09:53:27,789 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 448 ms to try 5000 different iterations.  Found a solution that moves 968 regions; Going from a computed imbalance of 0.016081653948323997 to a new imbalance of 6.6587088186101E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006982778595039731); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.010615195681147192); MoveCostFunction : (multiplier=7.0, imbalance=0.10755555555555556, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006982778595039731); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003916666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-16 09:53:27,792 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1447994151:79 , srv1364061727:80 , srv172732821:81 , srv1777579649:83 , srv1795134188:83 , srv2065733404:83 , srv277425780:83 , srv304683096:83 , srv498831413:83 , srv516515022:83 , srv892807162:83 , srv963089777:83 , srv1153045974:84 , srv1178174874:84 , srv1500683094:84 , srv1519375836:84 , srv1549257265:84 , srv1657040172:84 , srv1924644169:84 , srv2032011782:84 , srv444517227:84 , srv450541360:84 , srv509323606:84 , srv511398101:84 , srv901565069:84 , srv908342079:84 , srv910495922:84 , srv927748624:84 , srv1210395838:85 , srv1339147962:85 , srv1377958486:85 , srv1405523351:85 , srv1487260858:85 , srv1684531382:85 , srv1737195973:85 , srv1880172318:85 , srv1909170097:85 , srv2015208612:85 , srv2024031992:85 , srv2116685327:85 , srv2136825552:85 , srv254264596:85 , srv293934176:85 , srv335305788:85 , srv421881687:85 , srv529079762:85 , srv712477873:85 , srv931864120:85 , srv1004953103:86 , srv1050608744:86 , srv1067463010:86 , srv1183423838:86 , srv1281806801:86 , srv1316561937:86 , srv1399646197:86 , srv1502798044:86 , srv1522825815:86 , srv161438503:86 , srv1734357949:86 , srv1808550494:86 , srv2036466530:86 , srv2051106511:86 , srv329168658:86 , srv329901127:86 , srv359133873:86 , srv399935515:86 , srv407789080:86 , srv527198136:86 , srv531311072:86 , srv595837131:86 , srv689772699:86 , srv775035430:86 , srv842367805:86 , srv954018157:86 , srv995316746:86 , srv1140364989:87 , srv1385540086:87 , srv1872073586:87 , srv191998867:87 , srv2046627257:87 , srv375306944:87 , srv44171577:87 , srv46595306:87 , srv671283221:87 , srv695595066:87 , srv768278717:87 , srv83908066:87 , srv865297472:87 , srv91965146:87 , srv922632106:87 , srv930192434:87 , srv940020697:87 , srv1006595598:88 , srv1137114292:88 , srv1415722315:88 , srv1507788191:88 , srv1581004476:88 , srv1677549317:88 , srv17427872:88 , srv1643441367:552 }
2023-05-16 09:53:27,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-16 09:53:27,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1507788191=23, srv1500683094=21, srv1677549317=31, srv768278717=82, srv1909170097=42, srv527198136=74, srv695595066=80, srv910495922=90, srv2015208612=45, srv1067463010=3, srv1153045974=6, srv83908066=84, srv712477873=81, srv1183423838=8, srv172732821=33, srv940020697=96, srv375306944=62, srv2046627257=49, srv421881687=65, srv775035430=83, srv931864120=95, srv1519375836=24, srv1415722315=18, srv1880172318=41, srv46595306=69, srv1737195973=35, srv927748624=93, srv1405523351=17, srv329901127=59, srv671283221=78, srv511398101=72, srv91965146=91, srv1549257265=26, srv1643441367=29, srv1808550494=39, srv1316561937=11, srv2051106511=50, srv892807162=87, srv1924644169=44, srv191998867=43, srv450541360=68, srv359133873=61, srv1581004476=27, srv1522825815=25, srv304683096=57, srv1140364989=5, srv2136825552=53, srv1487260858=20, srv1377958486=14, srv595837131=77, srv1006595598=1, srv407789080=64, srv2065733404=51, srv954018157=97, srv930192434=94, srv963089777=98, srv335305788=60, srv1734357949=34, srv44171577=66, srv161438503=28, srv1447994151=19, srv922632106=92, srv2032011782=47, srv1281806801=10, srv498831413=70, srv1872073586=40, srv399935515=63, srv516515022=73, srv901565069=88, srv1684531382=32, srv1795134188=38, srv1210395838=9, srv17427872=36, srv1137114292=4, srv444517227=67, srv254264596=54, srv2036466530=48, srv995316746=99, srv1399646197=16, srv1339147962=12, srv1364061727=13, srv689772699=79, srv1657040172=30, srv1777579649=37, srv1385540086=15, srv2116685327=52, srv1502798044=22, srv1050608744=2, srv277425780=55, srv1004953103=0, srv293934176=56, srv329168658=58, srv2024031992=46, srv908342079=89, srv509323606=71, srv842367805=85, srv865297472=86, srv531311072=76, srv529079762=75, srv1178174874=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-16 09:53:27,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-16 09:53:27,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-16 09:53:27,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-16 09:53:27,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-16 09:53:27,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-16 09:53:27,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-16 09:53:27,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-16 09:53:27,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-16 09:53:27,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-16 09:53:27,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-16 09:53:27,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-16 09:53:27,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-16 09:53:27,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-16 09:53:27,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-16 09:53:27,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-16 09:53:27,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-16 09:53:27,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-16 09:53:27,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-16 09:53:27,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-16 09:53:27,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-16 09:53:27,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-16 09:53:27,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-16 09:53:27,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-16 09:53:27,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-16 09:53:27,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-16 09:53:27,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-16 09:53:27,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-16 09:53:27,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-16 09:53:27,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-16 09:53:27,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-16 09:53:27,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-16 09:53:27,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-16 09:53:27,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-16 09:53:27,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-16 09:53:27,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-16 09:53:27,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-16 09:53:27,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-16 09:53:27,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-16 09:53:27,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-16 09:53:27,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-16 09:53:27,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-16 09:53:27,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-16 09:53:27,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-16 09:53:27,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-16 09:53:27,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-16 09:53:27,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-16 09:53:27,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-16 09:53:27,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-16 09:53:27,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-16 09:53:27,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-16 09:53:27,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-16 09:53:27,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-16 09:53:27,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-16 09:53:27,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-16 09:53:27,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-16 09:53:27,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-16 09:53:27,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-16 09:53:27,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-16 09:53:27,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-16 09:53:27,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-16 09:53:27,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-16 09:53:27,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-16 09:53:27,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-16 09:53:27,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-16 09:53:27,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-16 09:53:27,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-16 09:53:27,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-16 09:53:27,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-16 09:53:27,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-16 09:53:27,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-16 09:53:27,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-16 09:53:27,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-16 09:53:27,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-16 09:53:27,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-16 09:53:27,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-16 09:53:27,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-16 09:53:27,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-16 09:53:27,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-16 09:53:27,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-16 09:53:27,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-16 09:53:27,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-16 09:53:27,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-16 09:53:27,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-16 09:53:27,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-16 09:53:27,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-16 09:53:27,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-16 09:53:27,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-16 09:53:27,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-16 09:53:27,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-16 09:53:27,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-16 09:53:27,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-16 09:53:27,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-16 09:53:27,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-16 09:53:27,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-16 09:53:27,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-16 09:53:27,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-16 09:53:27,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-16 09:53:27,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-16 09:53:27,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-16 09:53:27,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-16 09:53:27,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-16 09:53:28,013 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006982778595039731); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.010615195681147192); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006982778595039731); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003916666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-16 09:53:28,013 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-16 09:53:28,014 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=6.590928009901826E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006982778595039731); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.010615195681147192); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006982778595039731); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003916666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-16 09:53:28,334 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 468 ms to try 5000 different iterations.  Found a solution that moves 272 regions; Going from a computed imbalance of 6.590928009901826E-4 to a new imbalance of 2.2705164210150757E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.470732932414788E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0023688968483956715); MoveCostFunction : (multiplier=7.0, imbalance=0.030222222222222223, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.470732932414788E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-16 09:53:28,336 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1880172318:83 , srv2046627257:83 , srv671283221:83 , srv842367805:83 , srv865297472:83 , srv1004953103:84 , srv1183423838:84 , srv1385540086:84 , srv1399646197:84 , srv1447994151:84 , srv1500683094:84 , srv1502798044:84 , srv1507788191:84 , srv1581004476:84 , srv161438503:84 , srv1657040172:84 , srv1677549317:84 , srv17427872:84 , srv1795134188:84 , srv2015208612:84 , srv2036466530:84 , srv2116685327:84 , srv2136825552:84 , srv293934176:84 , srv329168658:84 , srv375306944:84 , srv44171577:84 , srv444517227:84 , srv450541360:84 , srv498831413:84 , srv509323606:84 , srv529079762:84 , srv595837131:84 , srv689772699:84 , srv901565069:84 , srv910495922:84 , srv940020697:84 , srv954018157:84 , srv995316746:84 , srv1006595598:85 , srv1050608744:85 , srv1137114292:85 , srv1140364989:85 , srv1153045974:85 , srv1178174874:85 , srv1210395838:85 , srv1281806801:85 , srv1316561937:85 , srv1339147962:85 , srv1364061727:85 , srv1377958486:85 , srv1405523351:85 , srv1415722315:85 , srv1487260858:85 , srv1519375836:85 , srv1522825815:85 , srv1684531382:85 , srv172732821:85 , srv1734357949:85 , srv1737195973:85 , srv1777579649:85 , srv1872073586:85 , srv1909170097:85 , srv191998867:85 , srv1924644169:85 , srv2024031992:85 , srv2032011782:85 , srv2065733404:85 , srv254264596:85 , srv277425780:85 , srv329901127:85 , srv335305788:85 , srv359133873:85 , srv399935515:85 , srv421881687:85 , srv46595306:85 , srv511398101:85 , srv516515022:85 , srv527198136:85 , srv531311072:85 , srv695595066:85 , srv712477873:85 , srv768278717:85 , srv775035430:85 , srv83908066:85 , srv892807162:85 , srv908342079:85 , srv91965146:85 , srv922632106:85 , srv930192434:85 , srv931864120:85 , srv963089777:85 , srv1067463010:86 , srv1549257265:86 , srv1808550494:86 , srv2051106511:86 , srv304683096:86 , srv407789080:86 , srv927748624:86 , srv1643441367:622 }
2023-05-16 09:53:28,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-16 09:53:28,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1507788191=23, srv1500683094=21, srv1677549317=31, srv768278717=82, srv1909170097=42, srv527198136=74, srv695595066=80, srv910495922=90, srv2015208612=45, srv1067463010=3, srv1153045974=6, srv83908066=84, srv712477873=81, srv1183423838=8, srv172732821=33, srv940020697=96, srv375306944=62, srv2046627257=49, srv421881687=65, srv775035430=83, srv931864120=95, srv1519375836=24, srv1415722315=18, srv1880172318=41, srv46595306=69, srv1737195973=35, srv927748624=93, srv1405523351=17, srv329901127=59, srv671283221=78, srv511398101=72, srv91965146=91, srv1549257265=26, srv1643441367=29, srv1808550494=39, srv1316561937=11, srv2051106511=50, srv892807162=87, srv1924644169=44, srv191998867=43, srv450541360=68, srv359133873=61, srv1581004476=27, srv1522825815=25, srv304683096=57, srv1140364989=5, srv2136825552=53, srv1487260858=20, srv1377958486=14, srv595837131=77, srv1006595598=1, srv407789080=64, srv2065733404=51, srv954018157=97, srv930192434=94, srv963089777=98, srv335305788=60, srv1734357949=34, srv44171577=66, srv161438503=28, srv1447994151=19, srv922632106=92, srv2032011782=47, srv1281806801=10, srv498831413=70, srv1872073586=40, srv399935515=63, srv516515022=73, srv901565069=88, srv1684531382=32, srv1795134188=38, srv1210395838=9, srv17427872=36, srv1137114292=4, srv444517227=67, srv254264596=54, srv2036466530=48, srv995316746=99, srv1399646197=16, srv1339147962=12, srv1364061727=13, srv689772699=79, srv1657040172=30, srv1777579649=37, srv1385540086=15, srv2116685327=52, srv1502798044=22, srv1050608744=2, srv277425780=55, srv1004953103=0, srv293934176=56, srv329168658=58, srv2024031992=46, srv908342079=89, srv509323606=71, srv842367805=85, srv865297472=86, srv531311072=76, srv529079762=75, srv1178174874=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-16 09:53:28,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-16 09:53:28,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-16 09:53:28,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-16 09:53:28,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-16 09:53:28,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-16 09:53:28,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-16 09:53:28,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-16 09:53:28,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-16 09:53:28,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-16 09:53:28,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-16 09:53:28,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-16 09:53:28,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-16 09:53:28,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-16 09:53:28,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-16 09:53:28,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-16 09:53:28,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-16 09:53:28,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-16 09:53:28,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-16 09:53:28,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-16 09:53:28,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-16 09:53:28,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-16 09:53:28,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-16 09:53:28,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-16 09:53:28,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-16 09:53:28,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-16 09:53:28,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-16 09:53:28,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-16 09:53:28,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-16 09:53:28,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-16 09:53:28,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-16 09:53:28,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-16 09:53:28,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-16 09:53:28,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-16 09:53:28,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-16 09:53:28,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-16 09:53:28,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-16 09:53:28,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-16 09:53:28,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-16 09:53:28,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-16 09:53:28,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-16 09:53:28,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-16 09:53:28,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-16 09:53:28,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-16 09:53:28,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-16 09:53:28,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-16 09:53:28,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-16 09:53:28,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-16 09:53:28,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-16 09:53:28,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-16 09:53:28,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-16 09:53:28,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-16 09:53:28,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-16 09:53:28,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-16 09:53:28,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-16 09:53:28,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-16 09:53:28,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-16 09:53:28,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-16 09:53:28,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-16 09:53:28,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-16 09:53:28,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-16 09:53:28,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-16 09:53:28,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-16 09:53:28,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-16 09:53:28,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-16 09:53:28,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-16 09:53:28,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-16 09:53:28,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-16 09:53:28,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-16 09:53:28,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-16 09:53:28,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-16 09:53:28,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-16 09:53:28,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-16 09:53:28,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-16 09:53:28,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-16 09:53:28,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-16 09:53:28,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-16 09:53:28,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-16 09:53:28,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-16 09:53:28,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-16 09:53:28,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-16 09:53:28,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-16 09:53:28,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-16 09:53:28,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-16 09:53:28,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-16 09:53:28,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-16 09:53:28,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-16 09:53:28,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-16 09:53:28,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-16 09:53:28,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-16 09:53:28,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-16 09:53:28,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-16 09:53:28,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-16 09:53:28,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-16 09:53:28,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-16 09:53:28,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-16 09:53:28,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-16 09:53:28,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-16 09:53:28,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-16 09:53:28,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-16 09:53:28,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-16 09:53:28,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-16 09:53:28,485 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=2.0800579502645554E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.470732932414788E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0023688968483956715); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.470732932414788E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-16 09:53:28,486 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1880172318:83 , srv2046627257:83 , srv671283221:83 , srv842367805:83 , srv865297472:83 , srv1004953103:84 , srv1183423838:84 , srv1385540086:84 , srv1399646197:84 , srv1447994151:84 , srv1500683094:84 , srv1502798044:84 , srv1507788191:84 , srv1581004476:84 , srv161438503:84 , srv1657040172:84 , srv1677549317:84 , srv17427872:84 , srv1795134188:84 , srv2015208612:84 , srv2036466530:84 , srv2116685327:84 , srv2136825552:84 , srv293934176:84 , srv329168658:84 , srv375306944:84 , srv44171577:84 , srv444517227:84 , srv450541360:84 , srv498831413:84 , srv509323606:84 , srv529079762:84 , srv595837131:84 , srv689772699:84 , srv901565069:84 , srv910495922:84 , srv940020697:84 , srv954018157:84 , srv995316746:84 , srv1006595598:85 , srv1050608744:85 , srv1137114292:85 , srv1140364989:85 , srv1153045974:85 , srv1178174874:85 , srv1210395838:85 , srv1281806801:85 , srv1316561937:85 , srv1339147962:85 , srv1364061727:85 , srv1377958486:85 , srv1405523351:85 , srv1415722315:85 , srv1487260858:85 , srv1519375836:85 , srv1522825815:85 , srv1684531382:85 , srv172732821:85 , srv1734357949:85 , srv1737195973:85 , srv1777579649:85 , srv1872073586:85 , srv1909170097:85 , srv191998867:85 , srv1924644169:85 , srv2024031992:85 , srv2032011782:85 , srv2065733404:85 , srv254264596:85 , srv277425780:85 , srv329901127:85 , srv335305788:85 , srv359133873:85 , srv399935515:85 , srv421881687:85 , srv46595306:85 , srv511398101:85 , srv516515022:85 , srv527198136:85 , srv531311072:85 , srv695595066:85 , srv712477873:85 , srv768278717:85 , srv775035430:85 , srv83908066:85 , srv892807162:85 , srv908342079:85 , srv91965146:85 , srv922632106:85 , srv930192434:85 , srv931864120:85 , srv963089777:85 , srv1067463010:86 , srv1549257265:86 , srv1808550494:86 , srv2051106511:86 , srv304683096:86 , srv407789080:86 , srv927748624:86 , srv1643441367:622 }
2023-05-16 09:53:28,516 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=264 (was 264), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=350 (was 350), ProcessCount=174 (was 172) - ProcessCount LEAK? -, AvailableMemoryMB=13074 (was 13964)
2023-05-16 09:53:28,534 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=264, MaxFileDescriptor=60000, SystemLoadAverage=350, ProcessCount=174, AvailableMemoryMB=13074
2023-05-16 09:53:28,535 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-16 09:53:28,535 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-05-16 09:53:28,549 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1008008917:84 , srv1058389956:84 , srv1149273935:84 , srv1248933327:84 , srv1257231123:84 , srv1303045142:84 , srv1415339541:84 , srv1485864000:84 , srv1560828929:84 , srv1575196608:84 , srv1598490051:84 , srv1640534757:84 , srv1653656508:84 , srv1744972957:84 , srv1785002106:84 , srv2011002583:84 , srv2111210118:84 , srv2139978241:84 , srv226336064:84 , srv367007310:84 , srv529820255:84 , srv542828784:84 , srv597727785:84 , srv702603093:84 , srv702995089:84 , srv830710847:84 , srv846259093:84 , srv868760732:84 , srv906907935:84 , srv970206318:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-05-16 09:53:28,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-16 09:53:28,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv830710847=25, srv906907935=28, srv1653656508=12, srv1008008917=0, srv1257231123=4, srv1248933327=3, srv1598490051=10, srv970206318=29, srv846259093=26, srv597727785=22, srv2011002583=15, srv529820255=20, srv1149273935=2, srv868760732=27, srv702603093=23, srv542828784=21, srv226336064=18, srv2111210118=16, srv2139978241=17, srv367007310=19, srv1560828929=8, srv1058389956=1, srv1575196608=9, srv1785002106=14, srv1744972957=13, srv1485864000=7, srv1640534757=11, srv1415339541=6, srv1303045142=5, srv702995089=24} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-16 09:53:28,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-16 09:53:28,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-16 09:53:28,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-16 09:53:28,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-16 09:53:28,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-16 09:53:28,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-16 09:53:28,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-16 09:53:28,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-16 09:53:28,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-16 09:53:28,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-16 09:53:28,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-16 09:53:28,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-16 09:53:28,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-16 09:53:28,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-16 09:53:28,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-16 09:53:28,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-16 09:53:28,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-16 09:53:28,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-16 09:53:28,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-16 09:53:28,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-16 09:53:28,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-16 09:53:28,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-16 09:53:28,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-16 09:53:28,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-16 09:53:28,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-16 09:53:28,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-16 09:53:28,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-16 09:53:28,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-16 09:53:28,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-16 09:53:28,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-16 09:53:28,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-05-16 09:53:28,588 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.14271020528984, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-16 09:53:28,589 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912642718461231, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.14271020528984, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-05-16 09:55:28,572 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 7032882 different iterations.  Found a solution that moves 1850 regions; Going from a computed imbalance of 0.9912642718461231 to a new imbalance of 4.317992290299789E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6851851851851852, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-16 09:55:28,579 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1008008917:90 , srv1058389956:90 , srv1149273935:90 , srv1248933327:90 , srv1257231123:90 , srv1303045142:90 , srv1415339541:90 , srv1485864000:90 , srv1560828929:90 , srv1575196608:90 , srv1598490051:90 , srv1640534757:90 , srv1653656508:90 , srv1744972957:90 , srv1785002106:90 , srv2011002583:90 , srv2111210118:90 , srv2139978241:90 , srv226336064:90 , srv367007310:90 , srv529820255:90 , srv542828784:90 , srv597727785:90 , srv702603093:90 , srv702995089:90 , srv830710847:90 , srv846259093:90 , srv868760732:90 , srv906907935:90 , srv970206318:90 }
2023-05-16 09:55:28,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-16 09:55:28,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv830710847=25, srv906907935=28, srv1653656508=12, srv1008008917=0, srv1257231123=4, srv1248933327=3, srv1598490051=10, srv970206318=29, srv846259093=26, srv597727785=22, srv2011002583=15, srv529820255=20, srv1149273935=2, srv868760732=27, srv702603093=23, srv542828784=21, srv226336064=18, srv2111210118=16, srv2139978241=17, srv367007310=19, srv1560828929=8, srv1058389956=1, srv1575196608=9, srv1785002106=14, srv1744972957=13, srv1485864000=7, srv1640534757=11, srv1415339541=6, srv1303045142=5, srv702995089=24} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-16 09:55:28,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-16 09:55:28,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-16 09:55:28,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-16 09:55:28,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-16 09:55:28,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-16 09:55:28,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-16 09:55:28,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-16 09:55:28,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-16 09:55:28,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-16 09:55:28,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-16 09:55:28,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-16 09:55:28,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-16 09:55:28,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-16 09:55:28,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-16 09:55:28,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-16 09:55:28,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-16 09:55:28,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-16 09:55:28,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-16 09:55:28,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-16 09:55:28,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-16 09:55:28,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-16 09:55:28,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-16 09:55:28,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-16 09:55:28,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-16 09:55:28,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-16 09:55:28,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-16 09:55:28,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-16 09:55:28,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-16 09:55:28,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-16 09:55:28,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-16 09:55:28,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-05-16 09:55:28,601 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-16 09:55:28,601 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1008008917:90 , srv1058389956:90 , srv1149273935:90 , srv1248933327:90 , srv1257231123:90 , srv1303045142:90 , srv1415339541:90 , srv1485864000:90 , srv1560828929:90 , srv1575196608:90 , srv1598490051:90 , srv1640534757:90 , srv1653656508:90 , srv1744972957:90 , srv1785002106:90 , srv2011002583:90 , srv2111210118:90 , srv2139978241:90 , srv226336064:90 , srv367007310:90 , srv529820255:90 , srv542828784:90 , srv597727785:90 , srv702603093:90 , srv702995089:90 , srv830710847:90 , srv846259093:90 , srv868760732:90 , srv906907935:90 , srv970206318:90 }
2023-05-16 09:55:28,617 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=264 (was 264), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=189 (was 350), ProcessCount=169 (was 174), AvailableMemoryMB=13035 (was 13074)